2024-03-07T07:07:40.523 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-07T07:07:40.524 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961 --name leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} --verbose -- /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/orig.config.yaml 2024-03-07T07:07:40.568 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T07:07:40.655 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961 branch: wip-lusov-qdb description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} email: null first_in_suite: false job_id: '7583961' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 kclient: syntax: v2 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 owner: scheduled_leonidus@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9221 sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2bcusr4TQ8ifT++ozEtIvUg06HaYDkl/6AuNcvne7Y4djc9CLlGseR6b6eKg71QjAdz5ePrAZ1Uh6d2DlYMC4= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJJs1ChUY4bEZKgqqgA0akQBieTGRewUeO2K2XEDMuldIsU4t355t0ku+Wta4XCmxCzmq+2kp7/L7lbr5KyBHDo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-07_05:46:12 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-07T07:07:40.655 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa; will attempt to use it 2024-03-07T07:07:40.656 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks 2024-03-07T07:07:40.656 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-07T07:07:40.657 INFO:teuthology.task.internal:Checking packages... 2024-03-07T07:07:40.683 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8b896a9e145796119f0451201dc4d53ddac97db0' 2024-03-07T07:07:40.683 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-07T07:07:40.683 INFO:teuthology.packaging:ref: None 2024-03-07T07:07:40.683 INFO:teuthology.packaging:tag: None 2024-03-07T07:07:40.683 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-07T07:07:40.683 INFO:teuthology.packaging:sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:07:40.683 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-07T07:07:40.910 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1848.g8b896a9e 2024-03-07T07:07:40.911 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-07T07:07:40.918 INFO:teuthology.task.internal:no buildpackages task found 2024-03-07T07:07:40.918 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-07T07:07:40.939 INFO:teuthology.task.internal:Saving configuration 2024-03-07T07:07:40.957 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-07T07:07:40.963 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T07:07:40.985 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:02:47.672493', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2bcusr4TQ8ifT++ozEtIvUg06HaYDkl/6AuNcvne7Y4djc9CLlGseR6b6eKg71QjAdz5ePrAZ1Uh6d2DlYMC4='} 2024-03-07T07:07:41.006 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:02:47.670462', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJJs1ChUY4bEZKgqqgA0akQBieTGRewUeO2K2XEDMuldIsU4t355t0ku+Wta4XCmxCzmq+2kp7/L7lbr5KyBHDo='} 2024-03-07T07:07:41.006 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-07T07:07:41.011 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-07T07:07:41.012 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-07T07:07:41.012 INFO:teuthology.run_tasks:Running task console_log... 2024-03-07T07:07:41.115 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6b367cff70>, signals=[15]) 2024-03-07T07:07:41.116 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-07T07:07:41.122 INFO:teuthology.task.internal:Opening connections... 2024-03-07T07:07:41.122 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:07:41.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:07:41.232 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:07:41.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:07:41.319 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-07T07:07:41.326 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-03-07T07:07:41.343 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-03-07T07:07:41.344 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-03-07T07:07:41.401 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-03-07T07:07:41.402 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T07:07:41.402 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-03-07T07:07:41.402 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T07:07:41.402 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T07:07:41.402 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T07:07:41.402 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-03-07T07:07:41.427 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-03-07T07:07:41.445 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-03-07T07:07:41.445 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-03-07T07:07:41.502 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T07:07:41.503 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T07:07:41.504 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-03-07T07:07:41.544 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-07T07:07:41.554 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-07T07:07:41.559 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-07T07:07:41.560 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-03-07T07:07:41.563 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-03-07T07:07:41.580 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-07T07:07:41.585 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-07T07:07:41.586 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-03-07T07:07:41.621 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-03-07T07:07:41.722 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-07T07:07:41.777 INFO:teuthology.run_tasks:Running task kernel... 2024-03-07T07:07:41.792 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-07T07:07:41.792 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-07T07:07:41.792 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-07T07:07:41.793 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-07T07:07:41.793 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-07T07:07:41.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:07:41.810 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-03-07T07:07:41.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:07:41.811 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-03-07T07:07:41.868 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-361.el9.x86_64 2024-03-07T07:07:41.868 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.14.0-361.el9.x86_64 2024-03-07T07:07:41.868 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum install -y kernel 2024-03-07T07:07:41.869 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-361.el9.x86_64 2024-03-07T07:07:41.870 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-361.el9.x86_64 2024-03-07T07:07:41.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-03-07T07:07:44.043 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-03-07T07:07:45.407 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS-9 - AppStream 4.3 MB/s | 8.1 MB 00:01 2024-03-07T07:07:45.785 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.5 MB 00:00 2024-03-07T07:07:47.041 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS-9 - BaseOS 8.1 MB/s | 2.5 MB 00:00 2024-03-07T07:07:47.117 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-03-07T07:07:48.065 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-07T07:07:50.025 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-07T07:07:51.034 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 19 MB 00:02 2024-03-07T07:07:55.743 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.1 MB 00:01 2024-03-07T07:07:56.017 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-03-07T07:07:57.236 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 15 kB 00:00 2024-03-07T07:07:57.659 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-03-07T07:08:02.315 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 21 MB 00:04 2024-03-07T07:08:07.512 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr:Modular dependency problems: 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 1: conflicting requests 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 2: conflicting requests 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 3: conflicting requests 2024-03-07T07:08:09.812 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 4: conflicting requests 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 5: conflicting requests 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 6: conflicting requests 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 7: conflicting requests 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 8: conflicting requests 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 9: conflicting requests 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 10: conflicting requests 2024-03-07T07:08:09.813 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 11: conflicting requests 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 12: conflicting requests 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 13: conflicting requests 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 14: conflicting requests 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 15: conflicting requests 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 16: conflicting requests 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 17: conflicting requests 2024-03-07T07:08:09.814 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 18: conflicting requests 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 19: conflicting requests 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 20: conflicting requests 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 21: conflicting requests 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 22: conflicting requests 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 23: conflicting requests 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 24: conflicting requests 2024-03-07T07:08:09.815 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 25: conflicting requests 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 26: conflicting requests 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 27: conflicting requests 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 28: conflicting requests 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 29: conflicting requests 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 30: conflicting requests 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.816 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 31: conflicting requests 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 32: conflicting requests 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 33: conflicting requests 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 34: conflicting requests 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 35: conflicting requests 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 36: conflicting requests 2024-03-07T07:08:09.817 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 37: conflicting requests 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 39: conflicting requests 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.818 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:08:09.819 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.820 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.821 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:08:09.821 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.821 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.821 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:08:09.821 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:08:10.369 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-07T07:08:10.369 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-07T07:08:10.479 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-07T07:08:10.485 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-03-07T07:08:10.486 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:08:10.487 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 100 M 2024-03-07T07:08:10.487 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 123 M 2024-03-07T07:08:10.487 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-07T07:08:13.729 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 1.9 MB/s | 6.0 MB 00:03 2024-03-07T07:08:18.695 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-07T07:08:23.659 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr:Modular dependency problems: 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 1: conflicting requests 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 2: conflicting requests 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 3: conflicting requests 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 4: conflicting requests 2024-03-07T07:08:25.903 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 5: conflicting requests 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 6: conflicting requests 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 7: conflicting requests 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 8: conflicting requests 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 9: conflicting requests 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 10: conflicting requests 2024-03-07T07:08:25.904 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 11: conflicting requests 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 12: conflicting requests 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 13: conflicting requests 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 14: conflicting requests 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 15: conflicting requests 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 16: conflicting requests 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.905 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 17: conflicting requests 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 18: conflicting requests 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 19: conflicting requests 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 20: conflicting requests 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 21: conflicting requests 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 22: conflicting requests 2024-03-07T07:08:25.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 23: conflicting requests 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 24: conflicting requests 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 25: conflicting requests 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 26: conflicting requests 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 27: conflicting requests 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 28: conflicting requests 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 29: conflicting requests 2024-03-07T07:08:25.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 30: conflicting requests 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 31: conflicting requests 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 32: conflicting requests 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 33: conflicting requests 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 34: conflicting requests 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 35: conflicting requests 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 36: conflicting requests 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 37: conflicting requests 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 39: conflicting requests 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:08:25.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-07T07:08:26.358 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-07T07:08:26.467 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-03-07T07:08:26.477 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-07T07:08:26.478 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:08:26.479 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-03-07T07:08:26.479 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-03-07T07:08:26.479 INFO:teuthology.orchestra.run.smithi120.stdout:Install 4 Packages 2024-03-07T07:08:26.479 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:08:26.479 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 100 M 2024-03-07T07:08:26.480 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 123 M 2024-03-07T07:08:26.480 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-03-07T07:08:33.611 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 925 kB/s | 21 MB 00:23 2024-03-07T07:08:34.726 INFO:teuthology.orchestra.run.smithi120.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 756 kB/s | 6.0 MB 00:08 2024-03-07T07:08:43.385 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 1.2 MB/s | 39 MB 00:32 2024-03-07T07:08:44.219 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 1.1 MB/s | 33 MB 00:30 2024-03-07T07:08:44.219 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:08:44.219 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.0 MB/s | 100 MB 00:33 2024-03-07T07:08:45.146 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-07T07:08:45.594 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-07T07:08:45.594 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-07T07:08:47.003 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-07T07:08:47.003 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-07T07:08:49.847 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-07T07:08:50.524 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-07T07:08:50.651 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:08:53.569 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:08:53.744 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:08:53.857 INFO:teuthology.orchestra.run.smithi120.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 783 kB/s | 21 MB 00:27 2024-03-07T07:08:58.878 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:08:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:07.972 INFO:teuthology.orchestra.run.smithi120.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 1.0 MB/s | 33 MB 00:33 2024-03-07T07:09:16.188 INFO:teuthology.orchestra.run.smithi120.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 812 kB/s | 39 MB 00:49 2024-03-07T07:09:16.189 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:09:16.189 INFO:teuthology.orchestra.run.smithi120.stdout:Total 2.0 MB/s | 100 MB 00:49 2024-03-07T07:09:17.070 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-03-07T07:09:17.458 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-03-07T07:09:17.458 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-03-07T07:09:18.812 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-03-07T07:09:18.812 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-03-07T07:09:21.770 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-03-07T07:09:22.424 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-07T07:09:22.576 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:09:25.386 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:09:25.581 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:09:30.322 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:09:30.650 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:34.914 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:34.914 INFO:teuthology.orchestra.run.smithi064.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-07T07:09:34.914 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:09:35.002 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:35.439 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:35.479 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:35.479 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-07T07:09:35.480 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:09:35.480 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:09:35.913 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-07T07:09:36.477 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-03-07T07:09:36.619 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-07T07:09:36.620 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-07T07:09:36.620 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-07T07:09:36.620 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-07T07:09:36.620 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-07T07:09:36.620 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-07T07:09:36.620 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-03-07T07:09:36.621 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-03-07T07:09:36.749 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-07T07:09:36.749 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-07T07:09:36.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-03-07T07:09:36.797 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-03-07T07:09:36.797 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-03-07T07:09:36.797 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-03-07T07:09:36.797 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 68.el9 2024-03-07T07:09:36.797 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8264254 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:09:36.798 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-07T07:09:36.799 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-07T07:09:36.799 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-07T07:09:36.799 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-03-07T07:09:36.799 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:09:36.799 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-03-07T07:09:36.800 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-07T07:09:36.800 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-427.el9.x86_64 2024-03-07T07:09:36.800 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-07T07:09:37.429 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-03-07T07:09:38.789 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-07T07:09:38.827 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-03-07T07:09:38.829 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-03-07T07:09:38.844 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.cHbkfHlkON 2024-03-07T07:09:38.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cHbkfHlkON 2024-03-07T07:09:38.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.cHbkfHlkON 2024-03-07T07:09:39.037 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.cHbkfHlkON is 6KB 2024-03-07T07:09:39.085 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.cHbkfHlkON 2024-03-07T07:09:39.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-03-07T07:09:39.168 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-07T07:09:39.168 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-07T07:09:39.168 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-07T07:09:39.168 INFO:teuthology.orchestra.run.smithi064.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-07T07:09:39.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-07T07:09:39.326 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-03-07T07:10:04.597 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:10:04.597 INFO:teuthology.orchestra.run.smithi120.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-07T07:10:04.597 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:10:04.682 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:10:05.192 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:10:05.192 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-07T07:10:05.192 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-07T07:10:05.192 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-07T07:10:05.631 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:10:05.632 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-03-07T07:10:06.160 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-03-07T07:10:06.307 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-07T07:10:06.307 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-07T07:10:06.307 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-07T07:10:06.308 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-07T07:10:06.308 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-07T07:10:06.308 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-07T07:10:06.308 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-03-07T07:10:06.308 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-03-07T07:10:06.428 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-07T07:10:06.429 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-07T07:10:06.429 DEBUG:teuthology.orchestra.run.smithi120:> sudo rpm -qi grub2-tools 2024-03-07T07:10:06.462 INFO:teuthology.orchestra.run.smithi120.stdout:Name : grub2-tools 2024-03-07T07:10:06.462 INFO:teuthology.orchestra.run.smithi120.stdout:Epoch : 1 2024-03-07T07:10:06.462 INFO:teuthology.orchestra.run.smithi120.stdout:Version : 2.06 2024-03-07T07:10:06.462 INFO:teuthology.orchestra.run.smithi120.stdout:Release : 68.el9 2024-03-07T07:10:06.462 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: x86_64 2024-03-07T07:10:06.462 INFO:teuthology.orchestra.run.smithi120.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Group : Unspecified 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Size : 8264254 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:License : GPLv3+ 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Packager : builder@centos.org 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Vendor : CentOS 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Summary : Support tools for GRUB. 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:Description : 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-07T07:10:06.463 INFO:teuthology.orchestra.run.smithi120.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-07T07:10:06.464 INFO:teuthology.orchestra.run.smithi120.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-07T07:10:06.464 INFO:teuthology.orchestra.run.smithi120.stdout:hardware devices. 2024-03-07T07:10:06.464 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:10:06.464 INFO:teuthology.orchestra.run.smithi120.stdout:This subpackage provides tools for support of all platforms. 2024-03-07T07:10:06.465 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-07T07:10:06.465 INFO:teuthology.task.kernel:Updating grub on smithi120 to boot 5.14.0-427.el9.x86_64 2024-03-07T07:10:06.465 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-07T07:10:07.149 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-03-07T07:10:08.519 INFO:teuthology.orchestra.run.smithi120.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-07T07:10:08.556 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-03-07T07:10:08.559 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-03-07T07:10:08.574 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.2FuveCg15P 2024-03-07T07:10:08.574 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2FuveCg15P 2024-03-07T07:10:08.642 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.2FuveCg15P 2024-03-07T07:10:08.785 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.2FuveCg15P is 6KB 2024-03-07T07:10:08.833 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.2FuveCg15P 2024-03-07T07:10:08.847 DEBUG:teuthology.orchestra.run.smithi120:> sudo /bin/ls /boot/loader/entries || true 2024-03-07T07:10:08.925 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-07T07:10:08.925 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-07T07:10:08.925 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-07T07:10:08.925 INFO:teuthology.orchestra.run.smithi120.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-07T07:10:08.926 DEBUG:teuthology.orchestra.run.smithi120:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-07T07:10:09.075 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-03-07T07:10:09.329 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-07T07:10:09.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-07T07:10:09.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:10:39.108 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-07T07:10:39.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-03-07T07:10:39.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:10:57.596 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-03-07T07:11:06.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-03-07T07:11:06.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:11:09.335 DEBUG:teuthology.orchestra.remote:timed out 2024-03-07T07:11:09.660 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.120 2024-03-07T07:11:18.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-07T07:11:18.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:11:18.730 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-03-07T07:11:19.131 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-07T07:11:19.132 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-07T07:11:19.132 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-03-07T07:11:19.145 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-427.el9.x86_64 2024-03-07T07:11:19.146 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-07T07:11:19.146 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-07T07:11:19.146 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-07T07:11:20.147 DEBUG:teuthology.parallel:result is None 2024-03-07T07:11:21.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-03-07T07:11:21.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:11:53.483 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-03-07T07:11:54.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-03-07T07:11:54.049 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-427.el9.x86_64"... 2024-03-07T07:11:54.049 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-03-07T07:11:54.069 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-427.el9.x86_64 2024-03-07T07:11:54.069 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-07T07:11:54.069 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-07T07:11:54.069 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-07T07:11:55.070 DEBUG:teuthology.parallel:result is None 2024-03-07T07:11:55.071 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-07T07:11:55.078 INFO:teuthology.task.internal:Creating test directory... 2024-03-07T07:11:55.079 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-07T07:11:55.082 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-07T07:11:55.099 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-07T07:11:55.105 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-07T07:11:55.116 INFO:teuthology.task.internal:Creating archive directory... 2024-03-07T07:11:55.116 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-07T07:11:55.141 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-07T07:11:55.348 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-07T07:11:55.357 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-07T07:11:55.357 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-07T07:11:55.360 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-07T07:11:55.468 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:11:55.489 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T07:11:55.639 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-07T07:11:55.649 INFO:teuthology.task.internal:Configuring sudo... 2024-03-07T07:11:55.649 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-07T07:11:55.652 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-07T07:11:55.683 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-07T07:11:55.692 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-07T07:11:55.693 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-07T07:11:55.726 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-07T07:11:55.742 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:11:55.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:11:55.877 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:11:55.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:11:56.003 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:11:56.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-07T07:11:56.071 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:11:56.116 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T07:11:56.171 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:11:56.251 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T07:11:56.292 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:11:56.293 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-07T07:11:56.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-03-07T07:11:56.360 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-03-07T07:11:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:11:56.434 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:11:56.892 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-07T07:11:56.900 INFO:teuthology.task.internal:Starting timer... 2024-03-07T07:11:56.900 INFO:teuthology.run_tasks:Running task pcp... 2024-03-07T07:11:56.912 INFO:teuthology.run_tasks:Running task selinux... 2024-03-07T07:11:56.920 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-03-07T07:11:56.977 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-03-07T07:11:56.978 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-03-07T07:11:57.030 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-03-07T07:11:57.032 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-07T07:11:57.032 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-03-07T07:11:57.056 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-03-07T07:11:57.056 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-03-07T07:11:57.077 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-03-07T07:11:57.078 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive'} 2024-03-07T07:11:57.078 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:11:57.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:11:57.125 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:11:57.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:11:57.154 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-07T07:11:57.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-03-07T07:11:57.200 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-03-07T07:11:57.229 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-07T07:11:57.239 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-07T07:11:57.256 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-07T07:11:57.257 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi120.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-07T07:16:19.899 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com')] 2024-03-07T07:16:19.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-07T07:16:19.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:16:19.992 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-03-07T07:16:20.074 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-07T07:16:20.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-03-07T07:16:20.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T07:16:20.162 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-03-07T07:16:20.247 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-03-07T07:16:20.247 INFO:teuthology.run_tasks:Running task clock... 2024-03-07T07:16:20.258 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-07T07:16:20.259 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-07T07:16:20.259 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:16:20.261 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-07T07:16:20.261 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:16:20.290 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-07T07:16:20.307 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-07T07:16:20.320 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-07T07:16:20.337 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-03-07T07:16:20.337 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-07T07:16:20.350 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-03-07T07:16:20.366 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-07T07:16:20.367 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-03-07T07:16:20.381 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-03-07T07:16:20.382 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-07T07:16:20.397 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-07T07:16:20.414 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-07T07:16:20.436 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:16:20.439 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:16:20.439 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-03-07T07:16:20.439 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.439 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.439 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.439 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.472 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:16:20.475 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:16:20.475 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-03-07T07:16:20.475 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.475 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.476 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.476 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:16:20.477 INFO:teuthology.run_tasks:Running task install... 2024-03-07T07:16:20.485 DEBUG:teuthology.task.install:project ceph 2024-03-07T07:16:20.485 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}} 2024-03-07T07:16:20.485 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'} 2024-03-07T07:16:20.486 INFO:teuthology.task.install:Using flavor: default 2024-03-07T07:16:20.491 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-07T07:16:20.491 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-07T07:16:20.492 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-07T07:16:20.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:16:20.494 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-07T07:16:20.494 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:16:20.636 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-07T07:16:20.637 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-07T07:16:20.740 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-07T07:16:20.740 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-07T07:16:20.747 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-07T07:16:20.747 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:16:20.747 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-07T07:16:20.775 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-07T07:16:20.775 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-07T07:16:20.844 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-07T07:16:20.850 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-07T07:16:20.850 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:16:20.850 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-07T07:16:20.878 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-07T07:16:20.879 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-07T07:16:20.923 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-07T07:16:20.946 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-07T07:16:20.989 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-03-07T07:16:20.991 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-03-07T07:16:21.026 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-07T07:16:21.091 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-03-07T07:16:21.093 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-03-07T07:16:21.238 INFO:teuthology.orchestra.run.smithi064.stdout:74 files removed 2024-03-07T07:16:21.265 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-07T07:16:21.325 INFO:teuthology.orchestra.run.smithi120.stdout:74 files removed 2024-03-07T07:16:21.352 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-07T07:16:22.056 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-03-07T07:16:22.132 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-03-07T07:16:22.440 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-03-07T07:16:22.557 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-07T07:16:22.815 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-03-07T07:16:22.950 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-03-07T07:16:23.301 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - AppStream 18 MB/s | 8.1 MB 00:00 2024-03-07T07:16:23.427 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS-9 - AppStream 18 MB/s | 8.1 MB 00:00 2024-03-07T07:16:25.051 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.5 MB 00:00 2024-03-07T07:16:25.136 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.5 MB 00:00 2024-03-07T07:16:26.476 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-03-07T07:16:26.509 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-03-07T07:16:29.052 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-07T07:16:40.578 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 783 kB/s | 6.1 MB 00:08 2024-03-07T07:16:42.245 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 15 kB 00:00 2024-03-07T07:16:42.833 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-07T07:16:43.413 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-07T07:16:47.767 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.1 MB 00:01 2024-03-07T07:16:48.413 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-07T07:16:49.460 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-03-07T07:16:50.620 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-07T07:16:50.905 INFO:teuthology.orchestra.run.smithi120.stderr:Modular dependency problems: 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 1: conflicting requests 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 2: conflicting requests 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 3: conflicting requests 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 4: conflicting requests 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 5: conflicting requests 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.906 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 6: conflicting requests 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 7: conflicting requests 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 8: conflicting requests 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 9: conflicting requests 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 10: conflicting requests 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 11: conflicting requests 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 12: conflicting requests 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.907 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 13: conflicting requests 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 14: conflicting requests 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 15: conflicting requests 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 16: conflicting requests 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 17: conflicting requests 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 18: conflicting requests 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 19: conflicting requests 2024-03-07T07:16:50.908 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 20: conflicting requests 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 21: conflicting requests 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 22: conflicting requests 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 23: conflicting requests 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 24: conflicting requests 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 25: conflicting requests 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.909 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 26: conflicting requests 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 27: conflicting requests 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 28: conflicting requests 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 29: conflicting requests 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 30: conflicting requests 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 31: conflicting requests 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 32: conflicting requests 2024-03-07T07:16:50.910 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 33: conflicting requests 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 34: conflicting requests 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 35: conflicting requests 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 36: conflicting requests 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 37: conflicting requests 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.911 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 39: conflicting requests 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.912 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.913 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.914 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:16:50.915 INFO:teuthology.orchestra.run.smithi120.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:16:51.350 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:16:51.351 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:16:51.363 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-07T07:16:51.364 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-07T07:16:51.365 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-07T07:16:51.366 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-07T07:16:51.368 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T07:16:51.371 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-07T07:16:51.372 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-07T07:16:51.465 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-03-07T07:16:51.485 INFO:teuthology.orchestra.run.smithi120.stdout:========================================================================================= 2024-03-07T07:16:51.485 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-03-07T07:16:51.485 INFO:teuthology.orchestra.run.smithi120.stdout:========================================================================================= 2024-03-07T07:16:51.485 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-07T07:16:51.486 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-07T07:16:51.487 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-03-07T07:16:51.488 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-07T07:16:51.489 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-03-07T07:16:51.490 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-07T07:16:51.491 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-03-07T07:16:51.492 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-07T07:16:51.493 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-07T07:16:51.494 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-03-07T07:16:51.495 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-03-07T07:16:51.496 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-03-07T07:16:51.497 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-03-07T07:16:51.498 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout:========================================================================================= 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout:Install 150 Packages 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 24 Packages 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 226 M 2024-03-07T07:16:51.499 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-03-07T07:16:52.547 INFO:teuthology.orchestra.run.smithi120.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-03-07T07:16:52.839 INFO:teuthology.orchestra.run.smithi120.stdout:(2/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 10 MB/s | 5.3 MB 00:00 2024-03-07T07:16:52.889 INFO:teuthology.orchestra.run.smithi120.stdout:(3/174): ceph-immutable-object-cache-19.0.0-184 2.9 MB/s | 145 kB 00:00 2024-03-07T07:16:53.040 INFO:teuthology.orchestra.run.smithi120.stdout:(4/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 15 MB/s | 2.3 MB 00:00 2024-03-07T07:16:53.148 INFO:teuthology.orchestra.run.smithi120.stdout:(5/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 14 MB/s | 1.5 MB 00:00 2024-03-07T07:16:53.173 INFO:teuthology.orchestra.run.smithi120.stdout:(6/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 1.4 MB/s | 889 kB 00:00 2024-03-07T07:16:53.407 INFO:teuthology.orchestra.run.smithi120.stdout:(7/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 18 MB/s | 4.7 MB 00:00 2024-03-07T07:16:53.924 INFO:teuthology.orchestra.run.smithi120.stdout:(8/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el9 19 MB/s | 9.8 MB 00:00 2024-03-07T07:16:53.974 INFO:teuthology.orchestra.run.smithi120.stdout:(9/174): ceph-selinux-19.0.0-1848.g8b896a9e.el9 507 kB/s | 25 kB 00:00 2024-03-07T07:16:55.778 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-07T07:16:56.374 INFO:teuthology.orchestra.run.smithi120.stdout:(10/174): ceph-common-19.0.0-1848.g8b896a9e.el9 5.0 MB/s | 20 MB 00:04 2024-03-07T07:16:56.808 INFO:teuthology.orchestra.run.smithi120.stdout:(11/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 17 MB/s | 47 MB 00:02 2024-03-07T07:16:56.833 INFO:teuthology.orchestra.run.smithi120.stdout:(12/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 481 kB/s | 220 kB 00:00 2024-03-07T07:16:56.858 INFO:teuthology.orchestra.run.smithi120.stdout:(13/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 543 kB/s | 27 kB 00:00 2024-03-07T07:16:56.892 INFO:teuthology.orchestra.run.smithi120.stdout:(14/174): libcephfs-devel-19.0.0-1848.g8b896a9e 534 kB/s | 31 kB 00:00 2024-03-07T07:16:56.942 INFO:teuthology.orchestra.run.smithi120.stdout:(15/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 8.7 MB/s | 746 kB 00:00 2024-03-07T07:16:56.967 INFO:teuthology.orchestra.run.smithi120.stdout:(16/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 2.1 MB/s | 163 kB 00:00 2024-03-07T07:16:56.992 INFO:teuthology.orchestra.run.smithi120.stdout:(17/174): librados-devel-19.0.0-1848.g8b896a9e. 2.5 MB/s | 127 kB 00:00 2024-03-07T07:16:57.143 INFO:teuthology.orchestra.run.smithi120.stdout:(18/174): libradosstriper1-19.0.0-1848.g8b896a9 2.8 MB/s | 501 kB 00:00 2024-03-07T07:16:57.219 INFO:teuthology.orchestra.run.smithi120.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 601 kB/s | 45 kB 00:00 2024-03-07T07:16:57.302 INFO:teuthology.orchestra.run.smithi120.stdout:(20/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 17 MB/s | 5.1 MB 00:00 2024-03-07T07:16:57.327 INFO:teuthology.orchestra.run.smithi120.stdout:(21/174): python3-ceph-common-19.0.0-1848.g8b89 1.2 MB/s | 131 kB 00:00 2024-03-07T07:16:57.353 INFO:teuthology.orchestra.run.smithi120.stdout:(22/174): python3-cephfs-19.0.0-1848.g8b896a9e. 3.1 MB/s | 161 kB 00:00 2024-03-07T07:16:57.395 INFO:teuthology.orchestra.run.smithi120.stdout:(23/174): python3-rados-19.0.0-1848.g8b896a9e.e 4.7 MB/s | 320 kB 00:00 2024-03-07T07:16:57.428 INFO:teuthology.orchestra.run.smithi120.stdout:(24/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 3.9 MB/s | 298 kB 00:00 2024-03-07T07:16:57.453 INFO:teuthology.orchestra.run.smithi120.stdout:(25/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 1.7 MB/s | 99 kB 00:00 2024-03-07T07:16:57.478 INFO:teuthology.orchestra.run.smithi120.stdout:(26/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-07T07:16:57.529 INFO:teuthology.orchestra.run.smithi120.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 3.3 MB/s | 170 kB 00:00 2024-03-07T07:16:57.588 INFO:teuthology.orchestra.run.smithi120.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 541 kB/s | 31 kB 00:00 2024-03-07T07:16:57.655 INFO:teuthology.orchestra.run.smithi120.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 2.0 MB/s | 138 kB 00:00 2024-03-07T07:16:57.713 INFO:teuthology.orchestra.run.smithi120.stdout:(30/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 12 MB/s | 3.1 MB 00:00 2024-03-07T07:16:57.789 INFO:teuthology.orchestra.run.smithi120.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 13 MB/s | 1.8 MB 00:00 2024-03-07T07:16:57.864 INFO:teuthology.orchestra.run.smithi120.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 3.2 MB/s | 248 kB 00:00 2024-03-07T07:16:57.922 INFO:teuthology.orchestra.run.smithi120.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 848 kB/s | 49 kB 00:00 2024-03-07T07:16:57.989 INFO:teuthology.orchestra.run.smithi120.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 246 kB/s | 16 kB 00:00 2024-03-07T07:16:58.064 INFO:teuthology.orchestra.run.smithi120.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 3.4 MB/s | 264 kB 00:00 2024-03-07T07:16:58.190 INFO:teuthology.orchestra.run.smithi120.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-03-07T07:16:58.224 INFO:teuthology.orchestra.run.smithi120.stdout:(37/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 4.6 MB/s | 748 kB 00:00 2024-03-07T07:16:58.249 INFO:teuthology.orchestra.run.smithi120.stdout:(38/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 312 kB/s | 18 kB 00:00 2024-03-07T07:16:58.324 INFO:teuthology.orchestra.run.smithi120.stdout:(39/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 176 kB/s | 18 kB 00:00 2024-03-07T07:16:58.383 INFO:teuthology.orchestra.run.smithi120.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 744 kB/s | 80 kB 00:00 2024-03-07T07:16:58.408 INFO:teuthology.orchestra.run.smithi064.stderr:Modular dependency problems: 2024-03-07T07:16:58.409 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:16:58.409 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 1: conflicting requests 2024-03-07T07:16:58.409 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.409 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 2: conflicting requests 2024-03-07T07:16:58.409 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.409 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 3: conflicting requests 2024-03-07T07:16:58.410 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.410 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 4: conflicting requests 2024-03-07T07:16:58.410 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.410 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 5: conflicting requests 2024-03-07T07:16:58.410 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.410 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 6: conflicting requests 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 7: conflicting requests 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 8: conflicting requests 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 9: conflicting requests 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 10: conflicting requests 2024-03-07T07:16:58.411 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.412 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 11: conflicting requests 2024-03-07T07:16:58.412 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.412 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 12: conflicting requests 2024-03-07T07:16:58.412 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.412 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 13: conflicting requests 2024-03-07T07:16:58.412 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 14: conflicting requests 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 15: conflicting requests 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 16: conflicting requests 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 17: conflicting requests 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.413 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 18: conflicting requests 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 19: conflicting requests 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 20: conflicting requests 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 21: conflicting requests 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 22: conflicting requests 2024-03-07T07:16:58.414 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 23: conflicting requests 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 24: conflicting requests 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 25: conflicting requests 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 26: conflicting requests 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 27: conflicting requests 2024-03-07T07:16:58.415 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 28: conflicting requests 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 29: conflicting requests 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 30: conflicting requests 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 31: conflicting requests 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.416 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 32: conflicting requests 2024-03-07T07:16:58.417 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.417 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 33: conflicting requests 2024-03-07T07:16:58.417 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.417 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 34: conflicting requests 2024-03-07T07:16:58.417 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.417 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 35: conflicting requests 2024-03-07T07:16:58.417 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 36: conflicting requests 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 37: conflicting requests 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:16:58.418 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 39: conflicting requests 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.419 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.420 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.420 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:16:58.420 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.420 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.420 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.420 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.421 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.422 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:16:58.423 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.423 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.423 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.423 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:16:58.449 INFO:teuthology.orchestra.run.smithi120.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-07T07:16:58.500 INFO:teuthology.orchestra.run.smithi120.stdout:(42/174): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-03-07T07:16:58.525 INFO:teuthology.orchestra.run.smithi120.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-03-07T07:16:58.551 INFO:teuthology.orchestra.run.smithi120.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 218 kB 00:00 2024-03-07T07:16:58.584 INFO:teuthology.orchestra.run.smithi120.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-03-07T07:16:58.626 INFO:teuthology.orchestra.run.smithi120.stdout:(46/174): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-03-07T07:16:58.651 INFO:teuthology.orchestra.run.smithi120.stdout:(47/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-03-07T07:16:58.677 INFO:teuthology.orchestra.run.smithi120.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-03-07T07:16:58.702 INFO:teuthology.orchestra.run.smithi120.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-07T07:16:58.736 INFO:teuthology.orchestra.run.smithi120.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-03-07T07:16:58.868 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:16:58.869 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T07:16:58.883 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-07T07:16:58.884 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-07T07:16:58.885 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-07T07:16:58.886 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-07T07:16:58.889 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T07:16:58.892 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-07T07:16:58.893 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-07T07:16:58.978 INFO:teuthology.orchestra.run.smithi120.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-03-07T07:16:58.990 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-07T07:16:59.003 INFO:teuthology.orchestra.run.smithi064.stdout:========================================================================================= 2024-03-07T07:16:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-03-07T07:16:59.003 INFO:teuthology.orchestra.run.smithi064.stdout:========================================================================================= 2024-03-07T07:16:59.003 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-07T07:16:59.003 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-07T07:16:59.004 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-07T07:16:59.005 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-07T07:16:59.006 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T07:16:59.007 INFO:teuthology.orchestra.run.smithi064.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-03-07T07:16:59.008 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-07T07:16:59.009 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-03-07T07:16:59.010 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-07T07:16:59.011 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-07T07:16:59.012 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-07T07:16:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-03-07T07:16:59.014 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-03-07T07:16:59.015 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-03-07T07:16:59.016 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout:========================================================================================= 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout:Install 150 Packages 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 24 Packages 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:16:59.017 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 226 M 2024-03-07T07:16:59.018 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-07T07:16:59.036 INFO:teuthology.orchestra.run.smithi120.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-03-07T07:16:59.070 INFO:teuthology.orchestra.run.smithi120.stdout:(53/174): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-03-07T07:16:59.095 INFO:teuthology.orchestra.run.smithi120.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 366 kB/s | 21 kB 00:00 2024-03-07T07:16:59.145 INFO:teuthology.orchestra.run.smithi120.stdout:(55/174): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-03-07T07:16:59.179 INFO:teuthology.orchestra.run.smithi120.stdout:(56/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-03-07T07:16:59.238 INFO:teuthology.orchestra.run.smithi120.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-03-07T07:16:59.296 INFO:teuthology.orchestra.run.smithi120.stdout:(58/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-03-07T07:16:59.355 INFO:teuthology.orchestra.run.smithi120.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-03-07T07:16:59.496 INFO:teuthology.orchestra.run.smithi064.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-03-07T07:16:59.513 INFO:teuthology.orchestra.run.smithi120.stdout:(60/174): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-03-07T07:16:59.589 INFO:teuthology.orchestra.run.smithi120.stdout:(61/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.7 MB/s | 3.0 MB 00:00 2024-03-07T07:16:59.639 INFO:teuthology.orchestra.run.smithi120.stdout:(62/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-03-07T07:16:59.681 INFO:teuthology.orchestra.run.smithi120.stdout:(63/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-03-07T07:16:59.715 INFO:teuthology.orchestra.run.smithi120.stdout:(64/174): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-03-07T07:16:59.765 INFO:teuthology.orchestra.run.smithi120.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-03-07T07:16:59.798 INFO:teuthology.orchestra.run.smithi120.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-03-07T07:16:59.982 INFO:teuthology.orchestra.run.smithi120.stdout:(67/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x8 2.5 MB/s | 17 MB 00:06 2024-03-07T07:16:59.999 INFO:teuthology.orchestra.run.smithi120.stdout:(68/174): openblas-0.3.21-2.el9.x86_64.rpm 150 kB/s | 35 kB 00:00 2024-03-07T07:17:00.249 INFO:teuthology.orchestra.run.smithi120.stdout:(69/174): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-07T07:17:00.296 INFO:teuthology.orchestra.run.smithi064.stdout:(2/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 1.1 MB/s | 889 kB 00:00 2024-03-07T07:17:00.408 INFO:teuthology.orchestra.run.smithi120.stdout:(70/174): postgresql-contrib-13.11-1.el9.x86_64 5.6 MB/s | 905 kB 00:00 2024-03-07T07:17:00.467 INFO:teuthology.orchestra.run.smithi120.stdout:(71/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 135 kB 00:00 2024-03-07T07:17:00.496 INFO:teuthology.orchestra.run.smithi064.stdout:(3/174): ceph-common-19.0.0-1848.g8b896a9e.el9. 17 MB/s | 20 MB 00:01 2024-03-07T07:17:00.513 INFO:teuthology.orchestra.run.smithi064.stdout:(4/174): ceph-immutable-object-cache-19.0.0-184 670 kB/s | 145 kB 00:00 2024-03-07T07:17:00.649 INFO:teuthology.orchestra.run.smithi064.stdout:(5/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 15 MB/s | 2.3 MB 00:00 2024-03-07T07:17:00.700 INFO:teuthology.orchestra.run.smithi120.stdout:(72/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.8 MB/s | 4.1 MB 00:00 2024-03-07T07:17:00.767 INFO:teuthology.orchestra.run.smithi120.stdout:(73/174): openblas-openmp-0.3.21-2.el9.x86_64.r 4.9 MB/s | 4.7 MB 00:00 2024-03-07T07:17:00.826 INFO:teuthology.orchestra.run.smithi120.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-03-07T07:17:00.893 INFO:teuthology.orchestra.run.smithi120.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-03-07T07:17:00.924 INFO:teuthology.orchestra.run.smithi064.stdout:(6/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-07T07:17:00.960 INFO:teuthology.orchestra.run.smithi120.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-07T07:17:01.018 INFO:teuthology.orchestra.run.smithi120.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-03-07T07:17:01.068 INFO:teuthology.orchestra.run.smithi120.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-03-07T07:17:01.127 INFO:teuthology.orchestra.run.smithi120.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-03-07T07:17:01.186 INFO:teuthology.orchestra.run.smithi120.stdout:(80/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-03-07T07:17:01.199 INFO:teuthology.orchestra.run.smithi064.stdout:(7/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-03-07T07:17:01.253 INFO:teuthology.orchestra.run.smithi120.stdout:(81/174): postgresql-server-13.11-1.el9.x86_64. 7.3 MB/s | 5.7 MB 00:00 2024-03-07T07:17:01.278 INFO:teuthology.orchestra.run.smithi120.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 378 kB/s | 35 kB 00:00 2024-03-07T07:17:01.337 INFO:teuthology.orchestra.run.smithi120.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-03-07T07:17:01.395 INFO:teuthology.orchestra.run.smithi120.stdout:(84/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-03-07T07:17:01.454 INFO:teuthology.orchestra.run.smithi120.stdout:(85/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-03-07T07:17:01.512 INFO:teuthology.orchestra.run.smithi120.stdout:(86/174): python3-prettytable-0.7.2-27.el9.noar 727 kB/s | 42 kB 00:00 2024-03-07T07:17:01.571 INFO:teuthology.orchestra.run.smithi120.stdout:(87/174): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-03-07T07:17:01.629 INFO:teuthology.orchestra.run.smithi120.stdout:(88/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-03-07T07:17:01.716 INFO:teuthology.orchestra.run.smithi064.stdout:(8/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 2.2 MB/s | 5.3 MB 00:02 2024-03-07T07:17:01.730 INFO:teuthology.orchestra.run.smithi120.stdout:(89/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-03-07T07:17:01.805 INFO:teuthology.orchestra.run.smithi120.stdout:(90/174): python3-babel-2.9.1-2.el9.noarch.rpm 6.1 MB/s | 6.0 MB 00:00 2024-03-07T07:17:01.822 INFO:teuthology.orchestra.run.smithi120.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 556 kB/s | 51 kB 00:00 2024-03-07T07:17:01.856 INFO:teuthology.orchestra.run.smithi120.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-03-07T07:17:01.892 INFO:teuthology.orchestra.run.smithi064.stdout:(9/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x86 17 MB/s | 17 MB 00:00 2024-03-07T07:17:01.906 INFO:teuthology.orchestra.run.smithi120.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-03-07T07:17:01.917 INFO:teuthology.orchestra.run.smithi064.stdout:(10/174): ceph-selinux-19.0.0-1848.g8b896a9e.el 125 kB/s | 25 kB 00:00 2024-03-07T07:17:01.990 INFO:teuthology.orchestra.run.smithi120.stdout:(94/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.9 MB/s | 5.1 MB 00:00 2024-03-07T07:17:02.006 INFO:teuthology.orchestra.run.smithi120.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-03-07T07:17:02.017 INFO:teuthology.orchestra.run.smithi064.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 2.2 MB/s | 220 kB 00:00 2024-03-07T07:17:02.048 INFO:teuthology.orchestra.run.smithi120.stdout:(96/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.6 MB/s | 216 kB 00:00 2024-03-07T07:17:02.065 INFO:teuthology.orchestra.run.smithi120.stdout:(97/174): qatlib-service-23.11.0-1.el9.x86_64.r 596 kB/s | 35 kB 00:00 2024-03-07T07:17:02.099 INFO:teuthology.orchestra.run.smithi120.stdout:(98/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-07T07:17:02.349 INFO:teuthology.orchestra.run.smithi120.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-03-07T07:17:02.366 INFO:teuthology.orchestra.run.smithi120.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 216 kB/s | 58 kB 00:00 2024-03-07T07:17:02.367 INFO:teuthology.orchestra.run.smithi064.stdout:(12/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 77 kB/s | 27 kB 00:00 2024-03-07T07:17:02.417 INFO:teuthology.orchestra.run.smithi120.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-07T07:17:02.434 INFO:teuthology.orchestra.run.smithi120.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-03-07T07:17:02.476 INFO:teuthology.orchestra.run.smithi120.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-03-07T07:17:02.476 INFO:teuthology.orchestra.run.smithi064.stdout:(13/174): libcephfs-devel-19.0.0-1848.g8b896a9e 289 kB/s | 31 kB 00:00 2024-03-07T07:17:02.526 INFO:teuthology.orchestra.run.smithi120.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-03-07T07:17:02.768 INFO:teuthology.orchestra.run.smithi120.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-03-07T07:17:02.818 INFO:teuthology.orchestra.run.smithi120.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-07T07:17:02.851 INFO:teuthology.orchestra.run.smithi064.stdout:(14/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 1.9 MB/s | 746 kB 00:00 2024-03-07T07:17:02.868 INFO:teuthology.orchestra.run.smithi120.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-03-07T07:17:02.952 INFO:teuthology.orchestra.run.smithi064.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 1.6 MB/s | 163 kB 00:00 2024-03-07T07:17:02.952 INFO:teuthology.orchestra.run.smithi120.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-03-07T07:17:03.010 INFO:teuthology.orchestra.run.smithi120.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-03-07T07:17:03.060 INFO:teuthology.orchestra.run.smithi064.stdout:(16/174): librados-devel-19.0.0-1848.g8b896a9e. 1.1 MB/s | 127 kB 00:00 2024-03-07T07:17:03.061 INFO:teuthology.orchestra.run.smithi120.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-03-07T07:17:03.161 INFO:teuthology.orchestra.run.smithi120.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-03-07T07:17:03.252 INFO:teuthology.orchestra.run.smithi064.stdout:(17/174): libradosstriper1-19.0.0-1848.g8b896a9 2.6 MB/s | 501 kB 00:00 2024-03-07T07:17:03.295 INFO:teuthology.orchestra.run.smithi120.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-03-07T07:17:03.352 INFO:teuthology.orchestra.run.smithi064.stdout:(18/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el 4.5 MB/s | 9.8 MB 00:02 2024-03-07T07:17:03.420 INFO:teuthology.orchestra.run.smithi120.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-03-07T07:17:03.427 INFO:teuthology.orchestra.run.smithi064.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 604 kB/s | 45 kB 00:00 2024-03-07T07:17:03.520 INFO:teuthology.orchestra.run.smithi120.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-03-07T07:17:03.561 INFO:teuthology.orchestra.run.smithi064.stdout:(20/174): python3-ceph-common-19.0.0-1848.g8b89 982 kB/s | 131 kB 00:00 2024-03-07T07:17:03.570 INFO:teuthology.orchestra.run.smithi120.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 872 kB/s | 43 kB 00:00 2024-03-07T07:17:03.621 INFO:teuthology.orchestra.run.smithi120.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-03-07T07:17:03.686 INFO:teuthology.orchestra.run.smithi064.stdout:(21/174): python3-cephfs-19.0.0-1848.g8b896a9e. 1.3 MB/s | 161 kB 00:00 2024-03-07T07:17:03.688 INFO:teuthology.orchestra.run.smithi120.stdout:(117/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 838 kB 00:00 2024-03-07T07:17:03.730 INFO:teuthology.orchestra.run.smithi120.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-07T07:17:03.755 INFO:teuthology.orchestra.run.smithi120.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-07T07:17:03.830 INFO:teuthology.orchestra.run.smithi120.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-03-07T07:17:03.889 INFO:teuthology.orchestra.run.smithi120.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 608 kB/s | 35 kB 00:00 2024-03-07T07:17:03.914 INFO:teuthology.orchestra.run.smithi120.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-03-07T07:17:03.945 INFO:teuthology.orchestra.run.smithi064.stdout:(22/174): python3-rados-19.0.0-1848.g8b896a9e.e 1.2 MB/s | 320 kB 00:00 2024-03-07T07:17:03.956 INFO:teuthology.orchestra.run.smithi120.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-07T07:17:04.132 INFO:teuthology.orchestra.run.smithi120.stdout:(124/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-03-07T07:17:04.157 INFO:teuthology.orchestra.run.smithi120.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 722 kB/s | 144 kB 00:00 2024-03-07T07:17:04.215 INFO:teuthology.orchestra.run.smithi120.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 304 kB/s | 18 kB 00:00 2024-03-07T07:17:04.241 INFO:teuthology.orchestra.run.smithi120.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-03-07T07:17:04.274 INFO:teuthology.orchestra.run.smithi120.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-03-07T07:17:04.300 INFO:teuthology.orchestra.run.smithi120.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-03-07T07:17:04.333 INFO:teuthology.orchestra.run.smithi120.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-03-07T07:17:04.387 INFO:teuthology.orchestra.run.smithi064.stdout:(23/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 19 MB/s | 47 MB 00:02 2024-03-07T07:17:04.392 INFO:teuthology.orchestra.run.smithi120.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 797 kB/s | 46 kB 00:00 2024-03-07T07:17:04.412 INFO:teuthology.orchestra.run.smithi064.stdout:(24/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 638 kB/s | 298 kB 00:00 2024-03-07T07:17:04.437 INFO:teuthology.orchestra.run.smithi064.stdout:(25/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 1.9 MB/s | 99 kB 00:00 2024-03-07T07:17:04.476 INFO:teuthology.orchestra.run.smithi120.stdout:(132/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-03-07T07:17:04.501 INFO:teuthology.orchestra.run.smithi120.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-03-07T07:17:04.518 INFO:teuthology.orchestra.run.smithi064.stdout:(26/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 807 kB/s | 85 kB 00:00 2024-03-07T07:17:04.526 INFO:teuthology.orchestra.run.smithi120.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-07T07:17:04.576 INFO:teuthology.orchestra.run.smithi120.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-07T07:17:04.610 INFO:teuthology.orchestra.run.smithi120.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-03-07T07:17:04.635 INFO:teuthology.orchestra.run.smithi120.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-07T07:17:04.638 INFO:teuthology.orchestra.run.smithi064.stdout:(27/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 3.7 MB/s | 5.1 MB 00:01 2024-03-07T07:17:04.661 INFO:teuthology.orchestra.run.smithi120.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-07T07:17:04.663 INFO:teuthology.orchestra.run.smithi064.stdout:(28/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 1.2 MB/s | 170 kB 00:00 2024-03-07T07:17:04.694 INFO:teuthology.orchestra.run.smithi120.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 485 kB/s | 28 kB 00:00 2024-03-07T07:17:04.719 INFO:teuthology.orchestra.run.smithi120.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 369 kB/s | 31 kB 00:00 2024-03-07T07:17:04.722 INFO:teuthology.orchestra.run.smithi064.stdout:(29/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 11 MB/s | 3.1 MB 00:00 2024-03-07T07:17:04.747 INFO:teuthology.orchestra.run.smithi064.stdout:(30/174): ceph-grafana-dashboards-19.0.0-1848.g 290 kB/s | 31 kB 00:00 2024-03-07T07:17:04.762 INFO:teuthology.orchestra.run.smithi120.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-07T07:17:04.764 INFO:teuthology.orchestra.run.smithi064.stdout:(31/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 1.3 MB/s | 138 kB 00:00 2024-03-07T07:17:04.787 INFO:teuthology.orchestra.run.smithi120.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-03-07T07:17:04.812 INFO:teuthology.orchestra.run.smithi120.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-03-07T07:17:04.837 INFO:teuthology.orchestra.run.smithi120.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-07T07:17:04.839 INFO:teuthology.orchestra.run.smithi064.stdout:(32/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 15 MB/s | 1.8 MB 00:00 2024-03-07T07:17:04.865 INFO:teuthology.orchestra.run.smithi064.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1848.g8b 2.4 MB/s | 248 kB 00:00 2024-03-07T07:17:04.870 INFO:teuthology.orchestra.run.smithi120.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-07T07:17:04.890 INFO:teuthology.orchestra.run.smithi064.stdout:(34/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 983 kB/s | 49 kB 00:00 2024-03-07T07:17:04.896 INFO:teuthology.orchestra.run.smithi120.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-07T07:17:04.921 INFO:teuthology.orchestra.run.smithi120.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-07T07:17:04.932 INFO:teuthology.orchestra.run.smithi064.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1848.g8 245 kB/s | 16 kB 00:00 2024-03-07T07:17:04.946 INFO:teuthology.orchestra.run.smithi120.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-03-07T07:17:04.957 INFO:teuthology.orchestra.run.smithi064.stdout:(36/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 3.8 MB/s | 264 kB 00:00 2024-03-07T07:17:04.971 INFO:teuthology.orchestra.run.smithi120.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-07T07:17:04.999 INFO:teuthology.orchestra.run.smithi064.stdout:(37/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 439 kB/s | 18 kB 00:00 2024-03-07T07:17:05.066 INFO:teuthology.orchestra.run.smithi064.stdout:(38/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 5.5 MB/s | 748 kB 00:00 2024-03-07T07:17:05.091 INFO:teuthology.orchestra.run.smithi064.stdout:(39/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 191 kB/s | 18 kB 00:00 2024-03-07T07:17:05.122 INFO:teuthology.orchestra.run.smithi120.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-03-07T07:17:05.184 INFO:teuthology.orchestra.run.smithi064.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 694 kB/s | 80 kB 00:00 2024-03-07T07:17:05.217 INFO:teuthology.orchestra.run.smithi064.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-07T07:17:05.275 INFO:teuthology.orchestra.run.smithi064.stdout:(42/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-03-07T07:17:05.309 INFO:teuthology.orchestra.run.smithi064.stdout:(43/174): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-03-07T07:17:05.314 INFO:teuthology.orchestra.run.smithi120.stdout:(151/174): NetworkManager-1.46.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-07T07:17:05.334 INFO:teuthology.orchestra.run.smithi064.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 218 kB 00:00 2024-03-07T07:17:05.359 INFO:teuthology.orchestra.run.smithi064.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-03-07T07:17:05.397 INFO:teuthology.orchestra.run.smithi120.stdout:(152/174): NetworkManager-libnm-1.46.0-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-03-07T07:17:05.401 INFO:teuthology.orchestra.run.smithi064.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-03-07T07:17:05.435 INFO:teuthology.orchestra.run.smithi064.stdout:(47/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-03-07T07:17:05.447 INFO:teuthology.orchestra.run.smithi120.stdout:(153/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-03-07T07:17:05.460 INFO:teuthology.orchestra.run.smithi064.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-03-07T07:17:05.473 INFO:teuthology.orchestra.run.smithi120.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 566 kB/s | 42 kB 00:00 2024-03-07T07:17:05.485 INFO:teuthology.orchestra.run.smithi064.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-07T07:17:05.515 INFO:teuthology.orchestra.run.smithi120.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-07T07:17:05.522 INFO:teuthology.orchestra.run.smithi064.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.4 MB/s | 215 kB 00:00 2024-03-07T07:17:05.557 INFO:teuthology.orchestra.run.smithi120.stdout:(156/174): elfutils-debuginfod-client-0.190-2.e 878 kB/s | 36 kB 00:00 2024-03-07T07:17:05.570 INFO:teuthology.orchestra.run.smithi064.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-03-07T07:17:05.582 INFO:teuthology.orchestra.run.smithi120.stdout:(157/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.8 MB/s | 248 kB 00:00 2024-03-07T07:17:05.607 INFO:teuthology.orchestra.run.smithi120.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-07T07:17:05.611 INFO:teuthology.orchestra.run.smithi064.stdout:(52/174): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-03-07T07:17:05.628 INFO:teuthology.orchestra.run.smithi064.stdout:(53/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-03-07T07:17:05.649 INFO:teuthology.orchestra.run.smithi120.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-03-07T07:17:05.654 INFO:teuthology.orchestra.run.smithi064.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 513 kB/s | 21 kB 00:00 2024-03-07T07:17:05.676 INFO:teuthology.orchestra.run.smithi120.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 345 kB/s | 23 kB 00:00 2024-03-07T07:17:05.679 INFO:teuthology.orchestra.run.smithi064.stdout:(55/174): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-03-07T07:17:05.700 INFO:teuthology.orchestra.run.smithi120.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-07T07:17:05.704 INFO:teuthology.orchestra.run.smithi064.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-03-07T07:17:05.725 INFO:teuthology.orchestra.run.smithi120.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-07T07:17:05.746 INFO:teuthology.orchestra.run.smithi064.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-03-07T07:17:05.750 INFO:teuthology.orchestra.run.smithi120.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-07T07:17:05.776 INFO:teuthology.orchestra.run.smithi120.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-07T07:17:05.796 INFO:teuthology.orchestra.run.smithi064.stdout:(58/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-07T07:17:05.801 INFO:teuthology.orchestra.run.smithi120.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-03-07T07:17:05.822 INFO:teuthology.orchestra.run.smithi064.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-03-07T07:17:05.847 INFO:teuthology.orchestra.run.smithi064.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-03-07T07:17:05.851 INFO:teuthology.orchestra.run.smithi120.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2024-03-07T07:17:05.883 INFO:teuthology.orchestra.run.smithi064.stdout:(61/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-03-07T07:17:05.914 INFO:teuthology.orchestra.run.smithi064.stdout:(62/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-03-07T07:17:05.939 INFO:teuthology.orchestra.run.smithi064.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-03-07T07:17:05.943 INFO:teuthology.orchestra.run.smithi120.stdout:(167/174): util-linux-2.37.4-18.el9.x86_64.rpm 25 MB/s | 2.3 MB 00:00 2024-03-07T07:17:05.965 INFO:teuthology.orchestra.run.smithi064.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 939 kB/s | 47 kB 00:00 2024-03-07T07:17:05.990 INFO:teuthology.orchestra.run.smithi064.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-03-07T07:17:05.993 INFO:teuthology.orchestra.run.smithi120.stdout:(168/174): librados2-19.0.0-1848.g8b896a9e.el9. 3.2 MB/s | 3.4 MB 00:01 2024-03-07T07:17:06.035 INFO:teuthology.orchestra.run.smithi120.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.2 MB/s | 2.2 MB 00:00 2024-03-07T07:17:06.069 INFO:teuthology.orchestra.run.smithi120.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-07T07:17:06.074 INFO:teuthology.orchestra.run.smithi064.stdout:(66/174): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2024-03-07T07:17:06.091 INFO:teuthology.orchestra.run.smithi064.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-03-07T07:17:06.094 INFO:teuthology.orchestra.run.smithi120.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-03-07T07:17:06.112 INFO:teuthology.orchestra.run.smithi120.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 264 kB/s | 31 kB 00:00 2024-03-07T07:17:06.116 INFO:teuthology.orchestra.run.smithi064.stdout:(68/174): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-03-07T07:17:06.154 INFO:teuthology.orchestra.run.smithi120.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-03-07T07:17:06.183 INFO:teuthology.orchestra.run.smithi064.stdout:(69/174): postgresql-13.11-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-03-07T07:17:06.204 INFO:teuthology.orchestra.run.smithi120.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 684 kB/s | 74 kB 00:00 2024-03-07T07:17:06.207 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:17:06.207 INFO:teuthology.orchestra.run.smithi120.stdout:Total 15 MB/s | 226 MB 00:14 2024-03-07T07:17:06.250 INFO:teuthology.orchestra.run.smithi064.stdout:(70/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 26 MB/s | 4.1 MB 00:00 2024-03-07T07:17:06.292 INFO:teuthology.orchestra.run.smithi064.stdout:(71/174): postgresql-contrib-13.11-1.el9.x86_64 8.2 MB/s | 905 kB 00:00 2024-03-07T07:17:06.317 INFO:teuthology.orchestra.run.smithi064.stdout:(72/174): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 135 kB 00:00 2024-03-07T07:17:06.376 INFO:teuthology.orchestra.run.smithi064.stdout:(73/174): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-03-07T07:17:06.451 INFO:teuthology.orchestra.run.smithi064.stdout:(74/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-07T07:17:06.510 INFO:teuthology.orchestra.run.smithi064.stdout:(75/174): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-03-07T07:17:06.535 INFO:teuthology.orchestra.run.smithi064.stdout:(76/174): python3-attrs-20.3.0-7.el9.noarch.rpm 550 kB/s | 87 kB 00:00 2024-03-07T07:17:06.560 INFO:teuthology.orchestra.run.smithi064.stdout:(77/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-03-07T07:17:06.585 INFO:teuthology.orchestra.run.smithi064.stdout:(78/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-03-07T07:17:06.602 INFO:teuthology.orchestra.run.smithi064.stdout:(79/174): python3-jsonpatch-1.21-16.el9.noarch. 609 kB/s | 26 kB 00:00 2024-03-07T07:17:06.627 INFO:teuthology.orchestra.run.smithi064.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 457 kB/s | 19 kB 00:00 2024-03-07T07:17:06.653 INFO:teuthology.orchestra.run.smithi064.stdout:(81/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-03-07T07:17:06.678 INFO:teuthology.orchestra.run.smithi064.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-03-07T07:17:06.770 INFO:teuthology.orchestra.run.smithi064.stdout:(83/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-03-07T07:17:06.796 INFO:teuthology.orchestra.run.smithi064.stdout:(84/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-03-07T07:17:06.821 INFO:teuthology.orchestra.run.smithi064.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-03-07T07:17:06.846 INFO:teuthology.orchestra.run.smithi064.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-03-07T07:17:06.871 INFO:teuthology.orchestra.run.smithi064.stdout:(87/174): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-03-07T07:17:06.896 INFO:teuthology.orchestra.run.smithi064.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-03-07T07:17:06.914 INFO:teuthology.orchestra.run.smithi064.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-03-07T07:17:06.947 INFO:teuthology.orchestra.run.smithi064.stdout:(90/174): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-03-07T07:17:06.972 INFO:teuthology.orchestra.run.smithi064.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-03-07T07:17:06.998 INFO:teuthology.orchestra.run.smithi064.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-03-07T07:17:07.064 INFO:teuthology.orchestra.run.smithi064.stdout:(93/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-03-07T07:17:07.090 INFO:teuthology.orchestra.run.smithi064.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-03-07T07:17:07.115 INFO:teuthology.orchestra.run.smithi064.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 875 kB/s | 44 kB 00:00 2024-03-07T07:17:07.141 INFO:teuthology.orchestra.run.smithi064.stdout:(96/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-07T07:17:07.166 INFO:teuthology.orchestra.run.smithi064.stdout:(97/174): qatlib-service-23.11.0-1.el9.x86_64.r 692 kB/s | 35 kB 00:00 2024-03-07T07:17:07.191 INFO:teuthology.orchestra.run.smithi064.stdout:(98/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-03-07T07:17:07.216 INFO:teuthology.orchestra.run.smithi064.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-03-07T07:17:07.242 INFO:teuthology.orchestra.run.smithi064.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-03-07T07:17:07.267 INFO:teuthology.orchestra.run.smithi064.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-07T07:17:07.292 INFO:teuthology.orchestra.run.smithi064.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-03-07T07:17:07.317 INFO:teuthology.orchestra.run.smithi064.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-03-07T07:17:07.393 INFO:teuthology.orchestra.run.smithi064.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-03-07T07:17:07.523 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-03-07T07:17:07.576 INFO:teuthology.orchestra.run.smithi064.stdout:(105/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-03-07T07:17:07.643 INFO:teuthology.orchestra.run.smithi064.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-03-07T07:17:07.841 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-03-07T07:17:07.841 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-03-07T07:17:07.885 INFO:teuthology.orchestra.run.smithi064.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-03-07T07:17:08.094 INFO:teuthology.orchestra.run.smithi064.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 782 kB/s | 548 kB 00:00 2024-03-07T07:17:08.119 INFO:teuthology.orchestra.run.smithi064.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 569 kB/s | 309 kB 00:00 2024-03-07T07:17:08.186 INFO:teuthology.orchestra.run.smithi064.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-03-07T07:17:08.228 INFO:teuthology.orchestra.run.smithi064.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-03-07T07:17:08.295 INFO:teuthology.orchestra.run.smithi064.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-03-07T07:17:08.345 INFO:teuthology.orchestra.run.smithi064.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-07T07:17:08.479 INFO:teuthology.orchestra.run.smithi064.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-03-07T07:17:08.537 INFO:teuthology.orchestra.run.smithi064.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-03-07T07:17:08.562 INFO:teuthology.orchestra.run.smithi064.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-03-07T07:17:08.630 INFO:teuthology.orchestra.run.smithi064.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-03-07T07:17:08.647 INFO:teuthology.orchestra.run.smithi064.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-03-07T07:17:08.663 INFO:teuthology.orchestra.run.smithi064.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-03-07T07:17:08.730 INFO:teuthology.orchestra.run.smithi064.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-07T07:17:08.755 INFO:teuthology.orchestra.run.smithi064.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-03-07T07:17:08.781 INFO:teuthology.orchestra.run.smithi064.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-03-07T07:17:08.814 INFO:teuthology.orchestra.run.smithi064.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-03-07T07:17:08.856 INFO:teuthology.orchestra.run.smithi064.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-07T07:17:08.907 INFO:teuthology.orchestra.run.smithi064.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-03-07T07:17:08.940 INFO:teuthology.orchestra.run.smithi064.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-03-07T07:17:09.016 INFO:teuthology.orchestra.run.smithi064.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-03-07T07:17:09.033 INFO:teuthology.orchestra.run.smithi064.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-03-07T07:17:09.049 INFO:teuthology.orchestra.run.smithi064.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-03-07T07:17:09.100 INFO:teuthology.orchestra.run.smithi064.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-07T07:17:09.142 INFO:teuthology.orchestra.run.smithi064.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-07T07:17:09.184 INFO:teuthology.orchestra.run.smithi064.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-03-07T07:17:09.209 INFO:teuthology.orchestra.run.smithi064.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-03-07T07:17:09.243 INFO:teuthology.orchestra.run.smithi064.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-03-07T07:17:09.276 INFO:teuthology.orchestra.run.smithi064.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 625 kB/s | 58 kB 00:00 2024-03-07T07:17:09.310 INFO:teuthology.orchestra.run.smithi064.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-03-07T07:17:09.335 INFO:teuthology.orchestra.run.smithi064.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-07T07:17:09.368 INFO:teuthology.orchestra.run.smithi064.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-03-07T07:17:09.402 INFO:teuthology.orchestra.run.smithi064.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-03-07T07:17:09.427 INFO:teuthology.orchestra.run.smithi064.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-03-07T07:17:09.461 INFO:teuthology.orchestra.run.smithi064.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-07T07:17:09.495 INFO:teuthology.orchestra.run.smithi064.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-03-07T07:17:09.520 INFO:teuthology.orchestra.run.smithi064.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-03-07T07:17:09.553 INFO:teuthology.orchestra.run.smithi064.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 552 kB/s | 51 kB 00:00 2024-03-07T07:17:09.595 INFO:teuthology.orchestra.run.smithi064.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-07T07:17:09.621 INFO:teuthology.orchestra.run.smithi064.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-03-07T07:17:09.654 INFO:teuthology.orchestra.run.smithi064.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-07T07:17:09.688 INFO:teuthology.orchestra.run.smithi064.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-03-07T07:17:09.721 INFO:teuthology.orchestra.run.smithi064.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-07T07:17:09.789 INFO:teuthology.orchestra.run.smithi064.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-03-07T07:17:09.972 INFO:teuthology.orchestra.run.smithi064.stdout:(151/174): NetworkManager-1.46.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-07T07:17:09.987 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-03-07T07:17:09.988 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-03-07T07:17:10.056 INFO:teuthology.orchestra.run.smithi064.stdout:(152/174): NetworkManager-libnm-1.46.0-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-03-07T07:17:10.089 INFO:teuthology.orchestra.run.smithi064.stdout:(153/174): NetworkManager-team-1.46.0-1.el9.x86 1.3 MB/s | 42 kB 00:00 2024-03-07T07:17:10.131 INFO:teuthology.orchestra.run.smithi064.stdout:(154/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.9 MB/s | 248 kB 00:00 2024-03-07T07:17:10.173 INFO:teuthology.orchestra.run.smithi064.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-07T07:17:10.206 INFO:teuthology.orchestra.run.smithi064.stdout:(156/174): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 36 kB 00:00 2024-03-07T07:17:10.273 INFO:teuthology.orchestra.run.smithi064.stdout:(157/174): librados2-19.0.0-1848.g8b896a9e.el9. 5.8 MB/s | 3.4 MB 00:00 2024-03-07T07:17:10.299 INFO:teuthology.orchestra.run.smithi064.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-07T07:17:10.332 INFO:teuthology.orchestra.run.smithi064.stdout:(159/174): libacl-2.3.1-4.el9.x86_64.rpm 697 kB/s | 23 kB 00:00 2024-03-07T07:17:10.374 INFO:teuthology.orchestra.run.smithi064.stdout:(160/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-07T07:17:10.399 INFO:teuthology.orchestra.run.smithi064.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-07T07:17:10.438 INFO:teuthology.orchestra.run.smithi064.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.4 MB/s | 155 kB 00:00 2024-03-07T07:17:10.492 INFO:teuthology.orchestra.run.smithi064.stdout:(163/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-03-07T07:17:10.509 INFO:teuthology.orchestra.run.smithi064.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 911 kB/s | 64 kB 00:00 2024-03-07T07:17:10.534 INFO:teuthology.orchestra.run.smithi064.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-03-07T07:17:10.568 INFO:teuthology.orchestra.run.smithi064.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 21 MB/s | 1.2 MB 00:00 2024-03-07T07:17:10.585 INFO:teuthology.orchestra.run.smithi064.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 311 kB/s | 29 kB 00:00 2024-03-07T07:17:10.652 INFO:teuthology.orchestra.run.smithi064.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 27 MB/s | 2.3 MB 00:00 2024-03-07T07:17:10.693 INFO:teuthology.orchestra.run.smithi064.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-07T07:17:10.710 INFO:teuthology.orchestra.run.smithi064.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 533 kB/s | 31 kB 00:00 2024-03-07T07:17:10.736 INFO:teuthology.orchestra.run.smithi064.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 323 kB 00:00 2024-03-07T07:17:10.769 INFO:teuthology.orchestra.run.smithi064.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 2.5 MB/s | 465 kB 00:00 2024-03-07T07:17:10.794 INFO:teuthology.orchestra.run.smithi064.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 74 kB 00:00 2024-03-07T07:17:10.828 INFO:teuthology.orchestra.run.smithi064.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.2 MB/s | 1.1 MB 00:00 2024-03-07T07:17:10.831 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-07T07:17:10.831 INFO:teuthology.orchestra.run.smithi064.stdout:Total 19 MB/s | 226 MB 00:11 2024-03-07T07:17:12.187 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-07T07:17:12.527 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-07T07:17:12.527 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-07T07:17:14.775 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-07T07:17:14.775 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-07T07:17:17.080 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-03-07T07:17:17.175 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-07T07:17:17.210 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-07T07:17:17.264 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:17:17.273 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:17:17.314 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-07T07:17:17.377 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-07T07:17:17.418 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-07T07:17:17.443 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-07T07:17:17.488 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-07T07:17:17.823 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-07T07:17:17.894 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:17:18.016 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:17:18.045 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:17:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:17:18.168 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:17:18.211 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-07T07:17:18.404 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-07T07:17:18.463 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-07T07:17:18.491 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:17:18.603 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:17:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:17:18.690 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:17:18.729 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-07T07:17:18.760 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:17:18.775 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:17:18.920 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:17:18.975 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:17:19.017 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-07T07:17:19.210 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-07T07:17:19.336 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:17:19.396 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:17:19.515 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:17:19.581 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:17:19.803 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:17:19.928 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-07T07:17:20.022 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-07T07:17:20.070 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-07T07:17:20.105 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-07T07:17:20.169 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-03-07T07:17:20.204 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-03-07T07:17:20.247 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-03-07T07:17:20.288 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-03-07T07:17:20.324 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-03-07T07:17:20.392 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-03-07T07:17:20.447 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-03-07T07:17:20.478 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-03-07T07:17:20.521 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-03-07T07:17:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-03-07T07:17:20.640 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-07T07:17:20.684 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-07T07:17:20.721 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-07T07:17:20.790 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-07T07:17:20.844 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-07T07:17:20.917 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-07T07:17:21.053 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-07T07:17:21.113 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-07T07:17:21.636 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-07T07:17:21.717 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-07T07:17:21.793 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-07T07:17:21.855 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:17:21.862 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:17:21.904 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-07T07:17:21.943 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-07T07:17:21.981 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-07T07:17:22.012 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-07T07:17:22.213 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-07T07:17:22.299 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-07T07:17:22.302 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-07T07:17:22.334 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-07T07:17:22.380 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:17:22.390 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T07:17:22.430 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-07T07:17:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-07T07:17:22.490 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-07T07:17:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-07T07:17:22.543 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-07T07:17:22.590 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-07T07:17:22.594 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-07T07:17:22.750 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-07T07:17:22.949 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-07T07:17:23.010 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:17:23.050 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-07T07:17:23.104 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-07T07:17:23.120 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:17:23.147 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:17:23.159 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-07T07:17:23.232 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-07T07:17:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:17:23.263 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-07T07:17:23.293 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:17:23.300 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-07T07:17:23.330 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-07T07:17:23.346 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-07T07:17:23.390 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-07T07:17:23.442 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-07T07:17:23.484 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-07T07:17:23.527 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-07T07:17:23.583 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-07T07:17:23.610 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:17:23.618 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-07T07:17:23.665 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-07T07:17:23.695 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-07T07:17:23.715 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:17:23.737 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-07T07:17:23.744 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:17:23.810 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T07:17:23.831 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-07T07:17:23.857 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-07T07:17:23.879 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-07T07:17:23.879 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:17:23.895 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T07:17:23.927 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-07T07:17:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-07T07:17:23.978 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-07T07:17:24.056 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-07T07:17:24.089 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:17:24.119 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T07:17:24.159 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-07T07:17:24.187 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-07T07:17:24.219 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-07T07:17:24.247 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-07T07:17:24.281 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-03-07T07:17:24.321 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-07T07:17:24.345 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-07T07:17:24.361 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-03-07T07:17:24.399 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-03-07T07:17:24.433 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-03-07T07:17:24.467 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-03-07T07:17:24.469 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:17:24.508 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-03-07T07:17:24.538 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T07:17:24.543 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-03-07T07:17:24.638 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-03-07T07:17:24.644 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:17:24.689 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:17:24.701 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-03-07T07:17:24.747 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-03-07T07:17:24.802 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-03-07T07:17:24.900 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-03-07T07:17:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T07:17:24.957 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-03-07T07:17:25.030 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-07T07:17:25.135 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-07T07:17:25.169 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-07T07:17:25.204 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-07T07:17:25.280 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-03-07T07:17:25.320 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-03-07T07:17:25.364 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-03-07T07:17:25.396 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-03-07T07:17:25.423 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-03-07T07:17:25.494 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-03-07T07:17:25.547 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-03-07T07:17:25.577 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-03-07T07:17:25.620 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-03-07T07:17:25.681 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-03-07T07:17:25.698 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-03-07T07:17:25.731 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-07T07:17:25.737 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-03-07T07:17:25.771 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-03-07T07:17:25.775 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-07T07:17:25.803 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-07T07:17:25.869 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-03-07T07:17:25.876 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-07T07:17:25.918 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-07T07:17:25.921 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-03-07T07:17:25.994 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-07T07:17:26.017 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-03-07T07:17:26.056 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-03-07T07:17:26.131 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-07T07:17:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-07T07:17:26.232 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-03-07T07:17:26.254 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-03-07T07:17:26.331 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-03-07T07:17:26.743 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-07T07:17:26.826 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-07T07:17:26.943 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-07T07:17:26.979 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-03-07T07:17:26.985 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:17:26.993 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T07:17:27.035 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-07T07:17:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-07T07:17:27.094 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-03-07T07:17:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-07T07:17:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-07T07:17:27.381 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-07T07:17:27.586 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-07T07:17:27.661 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-07T07:17:27.706 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-07T07:17:28.128 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-07T07:17:28.193 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-07T07:17:28.253 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-07T07:17:28.325 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-03-07T07:17:28.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-07T07:17:28.359 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-07T07:17:28.396 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-07T07:17:28.409 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-03-07T07:17:28.436 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-03-07T07:17:28.444 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-07T07:17:28.481 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-07T07:17:28.525 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-07T07:17:28.567 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-07T07:17:28.715 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-07T07:17:28.748 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-03-07T07:17:28.763 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-07T07:17:28.783 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:17:28.792 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-07T07:17:28.835 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-07T07:17:28.873 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:17:28.905 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 113/198 2024-03-07T07:17:28.939 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-07T07:17:28.941 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-03-07T07:17:28.975 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-07T07:17:29.016 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-07T07:17:29.041 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-07T07:17:29.066 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-07T07:17:29.146 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-07T07:17:29.270 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-07T07:17:29.307 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-07T07:17:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-07T07:17:29.369 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-03-07T07:17:29.393 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:17:29.420 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:17:29.425 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-07T07:17:29.465 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-03-07T07:17:29.486 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-03-07T07:17:29.490 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:17:29.512 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-03-07T07:17:29.513 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 117/198 2024-03-07T07:17:29.545 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-03-07T07:17:29.588 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-03-07T07:17:29.622 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-03-07T07:17:29.730 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-03-07T07:17:29.777 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-03-07T07:17:29.808 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-03-07T07:17:29.857 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-03-07T07:17:29.965 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-03-07T07:17:30.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-03-07T07:17:30.781 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-03-07T07:17:30.824 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-03-07T07:17:30.857 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-03-07T07:17:30.961 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-03-07T07:17:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-03-07T07:17:31.115 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-03-07T07:17:31.151 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-03-07T07:17:31.333 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-03-07T07:17:31.357 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-03-07T07:17:31.436 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-03-07T07:17:31.727 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:17:31.769 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:17:32.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-03-07T07:17:32.225 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-03-07T07:17:32.314 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:17:32.356 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:17:32.450 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:17:32.451 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-07T07:17:32.451 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-07T07:17:32.451 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:32.490 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:17:33.511 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-03-07T07:17:33.595 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-03-07T07:17:33.622 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-03-07T07:17:33.950 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-03-07T07:17:33.976 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:17:34.051 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T07:17:34.081 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 113/198 2024-03-07T07:17:34.118 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-03-07T07:17:34.596 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:17:34.621 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:17:34.676 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T07:17:34.698 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 117/198 2024-03-07T07:17:36.948 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:17:36.979 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:17:37.551 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T07:17:37.591 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:17:37.688 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T07:17:37.688 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-07T07:17:37.688 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-07T07:17:37.688 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:17:37.726 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-03-07T07:17:44.169 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:44.331 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:17:44.346 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:17:44.347 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:44.347 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-07T07:17:44.347 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:45.834 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:17:45.860 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:17:45.860 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:45.860 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-07T07:17:45.860 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:17:45.860 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:17:45.860 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:46.085 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:17:46.107 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:17:46.108 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:46.108 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-07T07:17:46.108 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:17:46.108 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:17:46.108 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:46.552 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:17:46.574 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:17:46.575 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:46.575 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-07T07:17:46.575 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:17:46.575 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:17:46.575 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:46.691 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-03-07T07:17:46.748 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-03-07T07:17:46.804 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-03-07T07:17:46.988 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-03-07T07:17:47.054 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-03-07T07:17:47.134 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-03-07T07:17:47.231 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-03-07T07:17:47.371 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-03-07T07:17:47.412 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 133/198 2024-03-07T07:17:47.445 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-03-07T07:17:47.476 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-03-07T07:17:47.513 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-03-07T07:17:47.548 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-03-07T07:17:47.891 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:17:47.922 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:17:47.954 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-07T07:17:48.015 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-07T07:17:48.037 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:17:48.264 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:17:48.288 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:17:49.256 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:17:49.297 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:17:49.399 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:17:49.562 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-07T07:17:49.591 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:17:49.614 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:17:49.614 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:49.614 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-07T07:17:49.614 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:17:49.615 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:17:49.615 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:49.658 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:17:49.669 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:17:49.763 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T07:17:49.764 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-03-07T07:17:49.764 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-03-07T07:17:49.764 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-03-07T07:17:49.764 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-03-07T07:17:49.764 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-03-07T07:17:49.764 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-03-07T07:17:49.765 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-03-07T07:17:49.765 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:17:49.768 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-07T07:17:49.945 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:17:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T07:17:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-07T07:17:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:17:51.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:17:51.524 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T07:17:51.524 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:51.524 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-07T07:17:51.524 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:17:51.524 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T07:17:51.524 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:17:51.749 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:17:51.771 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T07:17:51.771 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:51.771 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-07T07:17:51.772 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:17:51.772 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T07:17:51.772 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:17:52.233 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:17:52.255 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T07:17:52.255 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:52.255 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-07T07:17:52.255 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:17:52.255 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T07:17:52.256 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:17:52.372 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-03-07T07:17:52.438 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-03-07T07:17:52.502 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-03-07T07:17:52.693 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-03-07T07:17:52.742 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-03-07T07:17:52.831 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-03-07T07:17:52.928 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-03-07T07:17:53.077 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-03-07T07:17:53.117 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 133/198 2024-03-07T07:17:53.150 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-03-07T07:17:53.190 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-03-07T07:17:53.234 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-03-07T07:17:53.261 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-03-07T07:17:53.654 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:17:53.684 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T07:17:53.716 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-07T07:17:53.779 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-07T07:17:53.824 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:17:54.062 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T07:17:54.092 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:17:54.178 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 148/198 2024-03-07T07:17:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-07T07:17:55.100 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-03-07T07:17:55.103 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T07:17:55.124 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:17:55.134 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:17:55.146 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:17:55.146 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:55.147 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-07T07:17:55.147 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:17:55.147 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:17:55.147 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:55.192 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:17:55.213 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:17:55.213 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:55.213 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-07T07:17:55.213 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:55.238 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T07:17:55.259 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:17:55.281 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:17:55.281 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:55.281 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-07T07:17:55.281 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:17:55.281 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:17:55.281 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:55.404 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-07T07:17:55.428 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:17:55.450 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T07:17:55.450 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:55.450 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-07T07:17:55.451 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:17:55.451 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T07:17:55.451 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:17:55.495 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:17:55.506 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T07:17:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:17:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:17:55.598 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:17:55.599 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-07T07:17:55.599 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:17:55.599 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:17:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:17:55.607 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-07T07:17:55.651 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-07T07:17:55.681 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 156/198 2024-03-07T07:17:55.709 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-03-07T07:17:55.904 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 158/198 2024-03-07T07:17:55.951 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/198 2024-03-07T07:17:56.005 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-07T07:17:56.067 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-07T07:17:56.111 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-07T07:17:56.143 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-07T07:17:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-07T07:17:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-07T07:17:56.286 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-07T07:17:56.346 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-07T07:17:56.379 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-07T07:17:57.026 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:17:57.063 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:17:57.081 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:17:57.917 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-07T07:17:58.016 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-07T07:17:58.106 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-03-07T07:17:58.135 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-03-07T07:17:58.160 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-03-07T07:17:58.186 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-07T07:17:58.211 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-07T07:17:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-07T07:17:58.238 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:17:58.326 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:17:58.355 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-07T07:17:58.355 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:17:58.424 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:17:58.452 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-07T07:17:58.500 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-07T07:17:58.525 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-07T07:17:58.550 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-07T07:17:58.583 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-07T07:17:58.637 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-07T07:17:58.672 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-07T07:17:58.697 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:17:58.723 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-07T07:17:58.748 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:17:58.773 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-07T07:17:58.779 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:17:58.779 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:17:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:17:58.960 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:17:59.043 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:17:59.067 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-07T07:17:59.092 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-07T07:17:59.118 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-07T07:17:59.151 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-07T07:17:59.217 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:18:00.262 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 148/198 2024-03-07T07:18:00.327 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-07T07:18:01.229 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-03-07T07:18:01.261 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:18:01.284 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:18:01.284 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:18:01.284 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-07T07:18:01.284 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:18:01.284 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T07:18:01.284 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:01.328 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:18:01.350 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T07:18:01.350 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:18:01.350 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-07T07:18:01.350 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:01.395 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:18:01.418 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:18:01.418 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:18:01.418 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-07T07:18:01.418 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:18:01.418 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T07:18:01.418 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:01.738 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T07:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T07:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-07T07:18:01.760 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:18:01.761 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T07:18:01.761 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:01.813 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-07T07:18:01.843 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 156/198 2024-03-07T07:18:01.870 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-03-07T07:18:02.060 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 158/198 2024-03-07T07:18:02.118 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/198 2024-03-07T07:18:02.172 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-07T07:18:02.234 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-07T07:18:02.269 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-07T07:18:02.293 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-07T07:18:02.327 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-07T07:18:02.356 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-07T07:18:02.422 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-07T07:18:02.486 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-07T07:18:02.530 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-07T07:18:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:18:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:18:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T07:18:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-07T07:18:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-07T07:18:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-03-07T07:18:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-03-07T07:18:04.292 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-03-07T07:18:04.317 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-07T07:18:04.353 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-07T07:18:04.368 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:18:04.368 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-07T07:18:04.368 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-07T07:18:04.368 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-07T07:18:04.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-07T07:18:04.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-07T07:18:04.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-03-07T07:18:04.372 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-03-07T07:18:04.373 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-03-07T07:18:04.374 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/198 2024-03-07T07:18:04.375 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-03-07T07:18:04.376 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-03-07T07:18:04.377 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-07T07:18:04.378 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-07T07:18:04.379 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-07T07:18:04.380 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-07T07:18:04.381 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-07T07:18:04.382 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 155/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 157/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 159/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 161/198 2024-03-07T07:18:04.383 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-03-07T07:18:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-03-07T07:18:04.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-07T07:18:04.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-07T07:18:04.387 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-07T07:18:04.387 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:18:04.466 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T07:18:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-07T07:18:04.495 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:18:04.580 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T07:18:04.606 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-07T07:18:04.639 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-07T07:18:04.665 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-07T07:18:04.689 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-07T07:18:04.723 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-07T07:18:04.763 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-07T07:18:04.795 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-07T07:18:04.820 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:18:04.845 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-07T07:18:04.879 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:18:04.904 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-07T07:18:04.911 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:18:04.911 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:18:05.090 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T07:18:05.090 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:18:05.173 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T07:18:05.198 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-07T07:18:05.223 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-07T07:18:05.249 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-07T07:18:05.281 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-07T07:18:05.359 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:18:07.527 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-07T07:18:07.527 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:07.527 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-03-07T07:18:07.527 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:07.527 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:07.527 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:07.527 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: acl-2.3.1-4.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.528 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.529 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.530 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-07T07:18:07.531 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: jq-1.6-16.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-07T07:18:07.532 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-03-07T07:18:07.533 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.11-1.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-07T07:18:07.534 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-07T07:18:07.535 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-07T07:18:07.536 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-03-07T07:18:07.537 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-07T07:18:07.538 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-07T07:18:07.539 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-3.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-07T07:18:07.540 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-07T07:18:07.541 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:07.541 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-03-07T07:18:08.079 DEBUG:teuthology.parallel:result is None 2024-03-07T07:18:10.369 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T07:18:10.370 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-07T07:18:10.371 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-07T07:18:10.372 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-03-07T07:18:10.373 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-03-07T07:18:10.374 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-03-07T07:18:10.375 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-03-07T07:18:10.376 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-03-07T07:18:10.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-07T07:18:10.378 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-07T07:18:10.379 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-07T07:18:10.380 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 155/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 157/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 159/198 2024-03-07T07:18:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 161/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-03-07T07:18:10.382 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-03-07T07:18:10.383 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-07T07:18:10.384 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-07T07:18:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-07T07:18:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:13.420 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-03-07T07:18:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:13.420 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: acl-2.3.1-4.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-03-07T07:18:13.422 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.423 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-07T07:18:13.424 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-16.el9.x86_64 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-07T07:18:13.425 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-07T07:18:13.426 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.11-1.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-07T07:18:13.427 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-07T07:18:13.428 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-07T07:18:13.429 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-07T07:18:13.430 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-07T07:18:13.431 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-07T07:18:13.432 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-07T07:18:13.433 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-07T07:18:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-07T07:18:13.434 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:13.434 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-07T07:18:13.908 DEBUG:teuthology.parallel:result is None 2024-03-07T07:18:13.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:18:14.128 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-07T07:18:14.149 INFO:teuthology.orchestra.run.smithi064.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-07T07:18:14.150 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-07T07:18:14.150 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-07T07:18:14.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T07:18:14.365 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-07T07:18:14.392 INFO:teuthology.orchestra.run.smithi120.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-07T07:18:14.392 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-07T07:18:14.392 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-07T07:18:14.394 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-07T07:18:14.394 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:14.394 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-07T07:18:14.425 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:14.425 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-07T07:18:14.466 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-07T07:18:14.466 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:14.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-03-07T07:18:14.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-07T07:18:14.562 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:14.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-03-07T07:18:14.591 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-07T07:18:14.659 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-07T07:18:14.659 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:14.659 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-07T07:18:14.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-07T07:18:14.753 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:14.753 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-07T07:18:14.781 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-07T07:18:14.848 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-07T07:18:14.848 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:14.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-03-07T07:18:14.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-07T07:18:14.942 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:14.942 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-03-07T07:18:14.969 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-07T07:18:15.037 INFO:teuthology.run_tasks:Running task ceph... 2024-03-07T07:18:15.126 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-07T07:18:15.126 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2024-03-07T07:18:15.128 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /var/log/ceph 2024-03-07T07:18:15.154 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-07T07:18:15.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-07T07:18:15.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-07T07:18:15.221 INFO:tasks.ceph:Creating extra log directories... 2024-03-07T07:18:15.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-07T07:18:15.262 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-07T07:18:15.294 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-07T07:18:15.295 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-07T07:18:15.295 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-07T07:18:15.295 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:15.295 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-07T07:18:15.341 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-07T07:18:15.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-07T07:18:15.483 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:15.483 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-07T07:18:15.498 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-07T07:18:15.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-07T07:18:15.639 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-07T07:18:15.639 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-07T07:18:15.639 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961', 'branch': 'wip-lusov-qdb', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7583961', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9221, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'sleep_before_teardown': 0, 'subset': '111/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2bcusr4TQ8ifT++ozEtIvUg06HaYDkl/6AuNcvne7Y4djc9CLlGseR6b6eKg71QjAdz5ePrAZ1Uh6d2DlYMC4=', 'smithi120.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJJs1ChUY4bEZKgqqgA0akQBieTGRewUeO2K2XEDMuldIsU4t355t0ku+Wta4XCmxCzmq+2kp7/L7lbr5KyBHDo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-07_05:46:12', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-07T07:18:15.640 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-07T07:18:15.642 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-07T07:18:15.689 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2024-03-07T07:18:15.696 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/run/ceph 2024-03-07T07:18:15.756 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:15.756 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-03-07T07:18:15.775 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-07T07:18:15.775 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-03-07 07:18:05.690336502 +0000 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-03-07 07:15:52.195205642 +0000 2024-03-07T07:18:15.830 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-03-07 07:15:52.195205642 +0000 2024-03-07T07:18:15.831 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-03-07 07:15:52.195205642 +0000 2024-03-07T07:18:15.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-07T07:18:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-03-07T07:18:15.895 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-03-07T07:18:15.896 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.00025212 s, 2.0 MB/s 2024-03-07T07:18:15.896 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-07T07:18:15.952 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-03-07 07:18:05.690336502 +0000 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-03-07 07:15:52.553198944 +0000 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-03-07 07:15:52.553198944 +0000 2024-03-07T07:18:16.006 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-03-07 07:15:52.553198944 +0000 2024-03-07T07:18:16.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-07T07:18:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-03-07T07:18:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-03-07T07:18:16.072 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000211255 s, 2.4 MB/s 2024-03-07T07:18:16.073 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-07T07:18:16.128 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-03-07 07:18:05.690336502 +0000 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-03-07 07:15:52.894192563 +0000 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-03-07 07:15:52.894192563 +0000 2024-03-07T07:18:16.183 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-03-07 07:15:52.894192563 +0000 2024-03-07T07:18:16.184 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-07T07:18:16.247 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-03-07T07:18:16.247 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-03-07T07:18:16.247 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000180341 s, 2.8 MB/s 2024-03-07T07:18:16.248 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-07T07:18:16.304 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-03-07T07:18:16.358 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-03-07 07:18:05.691336484 +0000 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-03-07 07:15:53.237186145 +0000 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-03-07 07:15:53.237186145 +0000 2024-03-07T07:18:16.359 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-03-07 07:15:53.237186145 +0000 2024-03-07T07:18:16.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-07T07:18:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-03-07T07:18:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-03-07T07:18:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000195385 s, 2.6 MB/s 2024-03-07T07:18:16.424 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-07T07:18:16.479 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:16.480 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:16.480 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-03-07T07:18:16.496 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-07T07:18:16.497 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-03-07T07:18:16.552 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-03-07 07:17:59.585258156 +0000 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-03-07 07:15:52.174968040 +0000 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-03-07 07:15:52.174968040 +0000 2024-03-07T07:18:16.553 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-03-07 07:15:52.174968040 +0000 2024-03-07T07:18:16.553 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-07T07:18:16.619 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-03-07T07:18:16.619 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-03-07T07:18:16.619 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000209955 s, 2.4 MB/s 2024-03-07T07:18:16.620 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-07T07:18:16.677 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-03-07T07:18:16.731 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-03-07 07:17:59.586258133 +0000 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-03-07 07:15:52.530959988 +0000 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-03-07 07:15:52.530959988 +0000 2024-03-07T07:18:16.732 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-03-07 07:15:52.530959988 +0000 2024-03-07T07:18:16.733 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-07T07:18:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-03-07T07:18:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-03-07T07:18:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000204949 s, 2.5 MB/s 2024-03-07T07:18:16.798 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-07T07:18:16.854 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-03-07 07:17:59.586258133 +0000 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-03-07 07:15:52.883952005 +0000 2024-03-07T07:18:16.909 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-03-07 07:15:52.883952005 +0000 2024-03-07T07:18:16.910 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-03-07 07:15:52.883952005 +0000 2024-03-07T07:18:16.910 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-07T07:18:16.974 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-03-07T07:18:16.974 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-03-07T07:18:16.974 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000203658 s, 2.5 MB/s 2024-03-07T07:18:16.975 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-07T07:18:17.031 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-03-07T07:18:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-07T07:18:17.086 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T07:18:17.086 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-07T07:18:17.086 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T07:18:17.086 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T07:18:17.086 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-03-07 07:17:59.586258133 +0000 2024-03-07T07:18:17.087 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-03-07 07:15:53.245943817 +0000 2024-03-07T07:18:17.087 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-03-07 07:15:53.245943817 +0000 2024-03-07T07:18:17.087 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-03-07 07:15:53.245943817 +0000 2024-03-07T07:18:17.087 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-07T07:18:17.152 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-03-07T07:18:17.152 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-03-07T07:18:17.152 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000213762 s, 2.4 MB/s 2024-03-07T07:18:17.153 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-07T07:18:17.210 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:17.210 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-07T07:18:17.210 INFO:tasks.ceph:Generating config... 2024-03-07T07:18:17.211 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-07T07:18:17.212 INFO:tasks.ceph:[client] debug client = 20 2024-03-07T07:18:17.212 INFO:tasks.ceph:[client] debug ms = 1 2024-03-07T07:18:17.212 INFO:tasks.ceph:[client] pid file = 2024-03-07T07:18:17.212 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-07T07:18:17.212 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-07T07:18:17.212 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-07T07:18:17.212 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-07T07:18:17.212 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-07T07:18:17.212 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-07T07:18:17.212 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-07T07:18:17.213 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-07T07:18:17.214 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-07T07:18:17.214 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-07T07:18:17.214 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-07T07:18:17.214 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-07T07:18:17.214 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-07T07:18:17.214 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-07T07:18:17.215 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-07T07:18:17.215 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-07T07:18:17.215 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-07T07:18:17.215 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-07T07:18:17.215 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-07T07:18:17.215 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-07T07:18:17.215 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-07T07:18:17.216 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-07T07:18:17.216 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-07T07:18:17.216 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-07T07:18:17.216 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-07T07:18:17.217 INFO:tasks.ceph:Setting up mon.a... 2024-03-07T07:18:17.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-07T07:18:17.267 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.keyring 2024-03-07T07:18:17.270 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-07T07:18:17.315 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-07T07:18:17.343 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.64'), ('mon.b', '172.21.15.120'), ('mon.c', '[v2:172.21.15.120:3301,v1:172.21.15.120:6790]')] 2024-03-07T07:18:17.344 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.64,172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-07T07:18:17.345 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:17.345 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-07T07:18:17.397 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.64 --add b 172.21.15.120 --addv c '[v2:172.21.15.120:3301,v1:172.21.15.120:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:17.483 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:17.483 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: generated fsid 6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = quincy 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 0 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:fsid 6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:last_changed 2024-03-07T07:18:17.484881+0000 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-03-07T07:18:17.484881+0000 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:min_mon_release 17 (quincy) 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:election_strategy: 1 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:2: [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] mon.c 2024-03-07T07:18:17.484 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-07T07:18:17.485 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-07T07:18:17.500 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3... 2024-03-07T07:18:17.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-07T07:18:17.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-07T07:18:17.589 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2024-03-07T07:18:17.590 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2024-03-07T07:18:17.591 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2024-03-07T07:18:17.592 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.64,172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790]" 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on skipped message = False 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.593 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug trim objects = true 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2024-03-07T07:18:17.594 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore allocator = bitmap 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2024-03-07T07:18:17.595 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2024-03-07T07:18:17.596 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2024-03-07T07:18:17.597 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2024-03-07T07:18:17.598 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = "" 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2024-03-07T07:18:17.599 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds balancer = 20 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-03-07T07:18:17.600 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: chdir = "" 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: auth supported = cephx 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: filestore xattr use omap = true 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: mon clock drift allowed = 1.000 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: osd crush chooseleaf type = 0 2024-03-07T07:18:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: auth debug = true 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on old message = true 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on bug = true 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: mon pg warn max object skew = 0 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: # disable pg_autoscaler by default for new pools 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default size = 2 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow primary affinity = true 2024-03-07T07:18:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow pg remap = true 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on legacy crush tunables = false 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on crush straw calc version zero = false 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on no sortbitwise = false 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on osd down out interval zero = false 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on too few osds = false 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon_allow_pool_size_one = true 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: osd default data pool replay window = 5 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon allow pool delete = true 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: mon cluster log file level = debug 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: debug asserts on shutdown = true 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: mon health detail to clog = false 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: mon host = "172.21.15.64,172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790]" 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on skipped message = False 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = 6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout:[osd] 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: osd journal size = 100 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub load threshold = 5.0 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub max interval = 600 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: osd mclock profile = high_recovery_ops 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: osd recover clone overlap = true 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd recovery max chunk = 1048576 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug shutdown = true 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug op order = true 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify stray on activate = true 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug trim objects = true 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd open classes on start = true 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug pg log writeout = true 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd deep scrub update digest min age = 30 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: osd map max advance = 10 2024-03-07T07:18:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: journal zero on create = true 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: filestore ondisk finisher threads = 3 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: filestore apply finisher threads = 3 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: bdev debug aio = true 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug misdirected ops = true 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: bdev async discard = True 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: bdev enable discard = True 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore allocator = bitmap 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore block size = 96636764160 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore fsck on mount = True 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluefs = 1/20 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluestore = 1/20 2024-03-07T07:18:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: debug osd = 20 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: debug rocksdb = 4/10 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd backfillfull_ratio = 0.85 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd full ratio = 0.9 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd nearfull ratio = 0.8 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: osd failsafe full ratio = 0.95 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: osd objectstore = bluestore 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: osd op complaint time = 180 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr] 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: debug mgr = 20 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-03-07T07:18:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mgr/telemetry/nag = false 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout:[mon] 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: debug paxos = 20 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mon data avail warn = 5 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mon mgr mkfs grace = 240 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd reporter subtree level = osd 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd prime pg temp = true 2024-03-07T07:18:17.608 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: auth mon ticket ttl = 660 # 11m 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: auth service ticket ttl = 240 # 4m 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: # don't complain about insecure global_id in the test suite 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: # 1m isn't quite enough 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: mon_down_mkfs_grace = 2m 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_filestore_osds = false 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: mon op complaint time = 120 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-03-07T07:18:17.609 INFO:teuthology.orchestra.run.smithi120.stdout:[client] 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: rgw cache enabled = true 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable ops log = true 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable usage log = true 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: client mount timeout = 600 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: debug client = 20 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = "" 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: rados mon op timeout = 900 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout: rados osd op timeout = 900 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.a] 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.b] 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.c] 2024-03-07T07:18:17.610 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a] 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c] 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b] 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d] 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout:[mds] 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: debug mds = 20 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: debug mds balancer = 20 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: mds debug frag = True 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: mds debug scatterstat = True 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: mds op complaint time = 180 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: mds verify scatter = True 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: osd op complaint time = 180 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: rados mon op timeout = 900 2024-03-07T07:18:17.611 INFO:teuthology.orchestra.run.smithi120.stdout: rados osd op timeout = 900 2024-03-07T07:18:17.613 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-07T07:18:17.613 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-07T07:18:17.708 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-07T07:18:17.708 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:17.708 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-07T07:18:17.723 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:17.723 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-07T07:18:17.777 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:18:17.777 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:17.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-07T07:18:17.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-07T07:18:17.855 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:17.855 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:17.909 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:18:17.909 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:17.909 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-07T07:18:17.909 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-07T07:18:17.948 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:17.949 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:18.003 INFO:tasks.ceph:Setting up mon nodes... 2024-03-07T07:18:18.003 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-07T07:18:18.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-07T07:18:18.057 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-07T07:18:18.060 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-07T07:18:18.127 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-07T07:18:18.130 INFO:tasks.ceph:Setting up mds nodes... 2024-03-07T07:18:18.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-07T07:18:18.184 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-07T07:18:18.186 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-07T07:18:18.213 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-07T07:18:18.306 INFO:teuthology.orchestra.run.smithi064.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-07T07:18:18.309 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-07T07:18:18.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-07T07:18:18.390 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-07T07:18:18.392 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-07T07:18:18.419 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-07T07:18:18.512 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-07T07:18:18.515 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-07T07:18:18.541 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-07T07:18:18.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-07T07:18:18.583 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-07T07:18:18.598 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-07T07:18:18.681 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-07T07:18:18.697 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-07T07:18:18.739 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-07T07:18:18.753 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-07T07:18:18.836 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-07T07:18:18.850 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-07T07:18:18.851 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi064.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-07T07:18:18.851 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-07T07:18:18.877 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:18.878 INFO:tasks.ceph:role: osd.0 2024-03-07T07:18:18.878 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:18:18.878 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-07T07:18:18.951 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:18.952 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:19.443 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-03-07T07:18:19.445 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-03-07T07:18:19.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-07T07:18:19.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-07T07:18:19.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-07T07:18:19.658 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:19.658 INFO:tasks.ceph:role: osd.1 2024-03-07T07:18:19.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:18:19.658 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-07T07:18:19.732 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:19.732 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:19.732 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:19.732 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:19.733 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:19.733 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:19.733 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:20.193 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-03-07T07:18:20.195 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-03-07T07:18:20.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-07T07:18:20.270 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-07T07:18:20.342 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-07T07:18:20.409 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:20.409 INFO:tasks.ceph:role: osd.2 2024-03-07T07:18:20.409 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:18:20.409 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-07T07:18:20.483 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:20.483 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:20.483 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:20.483 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:20.483 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:20.483 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:20.484 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:20.484 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:20.484 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:20.484 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:20.966 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-03-07T07:18:20.968 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-03-07T07:18:20.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-07T07:18:21.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-07T07:18:21.100 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-07T07:18:21.167 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:21.167 INFO:tasks.ceph:role: osd.3 2024-03-07T07:18:21.167 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:18:21.167 DEBUG:teuthology.orchestra.run.smithi064:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-07T07:18:21.240 INFO:teuthology.orchestra.run.smithi064.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:21.240 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:21.240 INFO:teuthology.orchestra.run.smithi064.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:21.240 INFO:teuthology.orchestra.run.smithi064.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:21.240 INFO:teuthology.orchestra.run.smithi064.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:21.240 INFO:teuthology.orchestra.run.smithi064.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:21.241 INFO:teuthology.orchestra.run.smithi064.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:21.241 INFO:teuthology.orchestra.run.smithi064.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:21.241 INFO:teuthology.orchestra.run.smithi064.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:21.241 INFO:teuthology.orchestra.run.smithi064.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:21.726 INFO:teuthology.orchestra.run.smithi064.stdout:Discarding blocks...Done. 2024-03-07T07:18:21.727 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi064.front.sepia.ceph.com -o noatime 2024-03-07T07:18:21.728 DEBUG:teuthology.orchestra.run.smithi064:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-07T07:18:21.842 DEBUG:teuthology.orchestra.run.smithi064:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-07T07:18:21.875 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:21.972+0000 7f49cdd11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-07T07:18:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:21.972+0000 7f49cdd11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-07T07:18:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:21.972+0000 7f49cdd11640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-07T07:18:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:21.972+0000 7f49cdd11640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-07T07:18:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:21.972+0000 7f49cdd11640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-07T07:18:21.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:21.979+0000 7f49cdd11640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-07T07:18:24.831 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-07T07:18:24.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:24.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:24.952+0000 7efe784f5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-07T07:18:24.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:24.952+0000 7efe784f5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-07T07:18:24.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:24.952+0000 7efe784f5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-07T07:18:24.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:24.952+0000 7efe784f5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-07T07:18:24.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:24.953+0000 7efe784f5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-07T07:18:24.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:24.953+0000 7efe784f5640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-07T07:18:27.803 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-07T07:18:27.834 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:27.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:27.925+0000 7f75dd6c2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-07T07:18:27.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:27.925+0000 7f75dd6c2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-07T07:18:27.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:27.925+0000 7f75dd6c2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-07T07:18:27.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:27.925+0000 7f75dd6c2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-07T07:18:27.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:27.925+0000 7f75dd6c2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-07T07:18:27.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:27.926+0000 7f75dd6c2640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-07T07:18:30.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-07T07:18:30.801 DEBUG:teuthology.orchestra.run.smithi064:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:30.891+0000 7ff0fb20d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-07T07:18:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:30.891+0000 7ff0fb20d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-07T07:18:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:30.891+0000 7ff0fb20d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-07T07:18:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:30.892+0000 7ff0fb20d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-07T07:18:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:30.892+0000 7ff0fb20d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-07T07:18:30.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:30.892+0000 7ff0fb20d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-07T07:18:33.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-07T07:18:33.785 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-07T07:18:33.819 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:33.819 INFO:tasks.ceph:role: osd.4 2024-03-07T07:18:33.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:18:33.819 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-07T07:18:33.897 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:33.897 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:33.897 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:33.897 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:33.897 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:33.898 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:33.898 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:33.898 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:33.898 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:33.898 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:34.401 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-03-07T07:18:34.403 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-03-07T07:18:34.403 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-07T07:18:34.537 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-07T07:18:34.578 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-07T07:18:34.648 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:34.648 INFO:tasks.ceph:role: osd.5 2024-03-07T07:18:34.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:18:34.648 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:34.724 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:34.725 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:34.725 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:35.168 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-03-07T07:18:35.170 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-03-07T07:18:35.170 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-07T07:18:35.263 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-07T07:18:35.293 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-07T07:18:35.359 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:35.360 INFO:tasks.ceph:role: osd.6 2024-03-07T07:18:35.360 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:18:35.360 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-07T07:18:35.434 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:35.434 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:35.435 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:35.435 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:35.435 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:35.435 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:35.435 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:35.919 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-03-07T07:18:35.921 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-03-07T07:18:35.921 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-07T07:18:36.061 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-07T07:18:36.092 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-07T07:18:36.159 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T07:18:36.159 INFO:tasks.ceph:role: osd.7 2024-03-07T07:18:36.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:18:36.160 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-07T07:18:36.234 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T07:18:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T07:18:36.234 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T07:18:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T07:18:36.235 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T07:18:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-03-07T07:18:36.235 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T07:18:36.235 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T07:18:36.235 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T07:18:36.235 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T07:18:36.753 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-03-07T07:18:36.755 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-03-07T07:18:36.755 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-07T07:18:36.888 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-07T07:18:36.926 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:37.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:37.054+0000 7f440fc1f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-07T07:18:37.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:37.054+0000 7f440fc1f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-07T07:18:37.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:37.054+0000 7f440fc1f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-07T07:18:37.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:37.054+0000 7f440fc1f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-07T07:18:37.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:37.054+0000 7f440fc1f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-07T07:18:37.055 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:37.055+0000 7f440fc1f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-07T07:18:39.904 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-07T07:18:39.936 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:40.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:40.027+0000 7ff7024a6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-07T07:18:40.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:40.027+0000 7ff7024a6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-07T07:18:40.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:40.027+0000 7ff7024a6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-07T07:18:40.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:40.027+0000 7ff7024a6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-07T07:18:40.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:40.027+0000 7ff7024a6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-07T07:18:40.028 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:40.028+0000 7ff7024a6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-07T07:18:42.882 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-07T07:18:42.913 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:43.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:43.003+0000 7fcf6bfbd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-07T07:18:43.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:43.004+0000 7fcf6bfbd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-07T07:18:43.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:43.004+0000 7fcf6bfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-07T07:18:43.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:43.004+0000 7fcf6bfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-07T07:18:43.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:43.004+0000 7fcf6bfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-07T07:18:43.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:43.004+0000 7fcf6bfbd640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-07T07:18:45.641 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:18:45.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:18:45.865 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-07T07:18:45.896 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:45.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:45.987+0000 7f5954417640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-07T07:18:45.987 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:45.987+0000 7f5954417640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-07T07:18:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:45.987+0000 7f5954417640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-07T07:18:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:45.987+0000 7f5954417640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-07T07:18:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:45.987+0000 7f5954417640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-07T07:18:45.988 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:45.988+0000 7f5954417640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-07T07:18:48.847 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-07T07:18:48.879 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-07T07:18:48.879 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:48.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-07T07:18:48.911 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:48.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-07T07:18:48.986 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:48.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-07T07:18:49.058 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:49.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-07T07:18:49.128 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:49.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-07T07:18:49.196 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:49.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-07T07:18:49.264 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:49.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-07T07:18:49.331 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.332 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-07T07:18:49.363 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.363 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-07T07:18:49.432 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.432 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-07T07:18:49.502 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-07T07:18:49.570 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.570 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-07T07:18:49.638 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.638 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-07T07:18:49.705 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-07T07:18:49.773 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:49.773 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-07T07:18:49.789 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:49.789 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-07T07:18:49.846 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.846 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-07T07:18:49.861 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:49.861 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-07T07:18:49.916 INFO:tasks.ceph:Adding keys to all mons... 2024-03-07T07:18:49.916 DEBUG:teuthology.orchestra.run.smithi064:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-07T07:18:49.919 DEBUG:teuthology.orchestra.run.smithi120:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-07T07:18:49.960 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.y] 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullO6iEAxAAywLBdQ3VTqByWJiCPRhFyA== 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullxsIRCxAA04BZ4y8SMy9m3r4wg+jXXQ== 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aull3ftcEhAAvnqepJ2iz2sZeEEklpkK6A== 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.0] 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA9aulloo3+ORAAYhShJsI7HJTH8aSAwHOg7w== 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.1] 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBAaull+GDlOBAA5veEdAk2Q1YWzJJZo7tEHQ== 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.2] 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBDaullFa1DNxAAdfo86r9Id+f0zqBmYTrzxw== 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.3] 2024-03-07T07:18:49.961 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBGaullHtAyNRAAmQ3RllAvydkWIV44PJAyAg== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr.x] 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullyaSuBxAAK+chNI5foXZF/ihl065jgg== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aull7hZXFxAARYJ9stnsXcxHajcUfWv9XQ== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d] 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullv/CgHhAAp/jpJGTFbzFhBZQWJaTNFw== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.4] 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBNaullHB9RAxAAReqzx+tbp2+wRpM+eyMezg== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.5] 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBQaullAtK0ARAAxnNjtvkqTw3ycbUgMpw5+g== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.6] 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBTaull11tLABAAwQfZk+VvUp1XzZ081/TjNg== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[osd.7] 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBVaull5sjqOhAABBkugq1DUhRGzAkMPedDFg== 2024-03-07T07:18:49.962 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-03-07T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullg6DUIhAAM9uoMC2xuAIACpfgwkhF9A== 2024-03-07T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout:[client.2] 2024-03-07T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullpla4KBAAMWGeahYoOSfEqSfKAQrh/w== 2024-03-07T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout:[client.1] 2024-03-07T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullyVYaLBAAMjThSrGPA+wqxy1z0EWJzw== 2024-03-07T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout:[client.3] 2024-03-07T07:18:49.963 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullyYLnMRAAUIhgyc3Dt/FrY6BgPlfpBg== 2024-03-07T07:18:49.983 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.y] 2024-03-07T07:18:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullO6iEAxAAywLBdQ3VTqByWJiCPRhFyA== 2024-03-07T07:18:49.983 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a] 2024-03-07T07:18:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullxsIRCxAA04BZ4y8SMy9m3r4wg+jXXQ== 2024-03-07T07:18:49.983 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c] 2024-03-07T07:18:49.983 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aull3ftcEhAAvnqepJ2iz2sZeEEklpkK6A== 2024-03-07T07:18:49.983 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.0] 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA9aulloo3+ORAAYhShJsI7HJTH8aSAwHOg7w== 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.1] 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBAaull+GDlOBAA5veEdAk2Q1YWzJJZo7tEHQ== 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.2] 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBDaullFa1DNxAAdfo86r9Id+f0zqBmYTrzxw== 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.3] 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBGaullHtAyNRAAmQ3RllAvydkWIV44PJAyAg== 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.x] 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullyaSuBxAAK+chNI5foXZF/ihl065jgg== 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b] 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aull7hZXFxAARYJ9stnsXcxHajcUfWv9XQ== 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d] 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullv/CgHhAAp/jpJGTFbzFhBZQWJaTNFw== 2024-03-07T07:18:49.984 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.4] 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBNaullHB9RAxAAReqzx+tbp2+wRpM+eyMezg== 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.5] 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBQaullAtK0ARAAxnNjtvkqTw3ycbUgMpw5+g== 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.6] 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBTaull11tLABAAwQfZk+VvUp1XzZ081/TjNg== 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.7] 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBVaull5sjqOhAABBkugq1DUhRGzAkMPedDFg== 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullg6DUIhAAM9uoMC2xuAIACpfgwkhF9A== 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout:[client.2] 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullpla4KBAAMWGeahYoOSfEqSfKAQrh/w== 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout:[client.1] 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullyVYaLBAAMjThSrGPA+wqxy1z0EWJzw== 2024-03-07T07:18:49.985 INFO:teuthology.orchestra.run.smithi120.stdout:[client.3] 2024-03-07T07:18:49.986 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6aullyYLnMRAAUIhgyc3Dt/FrY6BgPlfpBg== 2024-03-07T07:18:49.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T07:18:50.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T07:18:50.080 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.138 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.140 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.194 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.197 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.248 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.250 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.303 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.357 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T07:18:50.410 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T07:18:50.462 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.464 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.518 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.521 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T07:18:50.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.577 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.628 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.631 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.686 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.741 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T07:18:50.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:50.794 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:50.848 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:50.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:50.906 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:50.909 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:50.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:50.965 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T07:18:51.019 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-07T07:18:51.019 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-07T07:18:51.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-07T07:18:51.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-07T07:18:51.168 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-07T07:18:51.195 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-07T07:18:51.292 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-07T07:18:51.319 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-07T07:18:51.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-07T07:18:51.482 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-07T07:18:51.511 DEBUG:teuthology.orchestra.run.smithi064:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:51.513 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-07T07:18:51.563 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-07T07:18:51.564 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-07T07:18:51.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-07T07:18:51.569 INFO:tasks.ceph.mon.a:Started 2024-03-07T07:18:51.569 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-07T07:18:51.569 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-07T07:18:51.606 INFO:tasks.ceph.mon.b:Started 2024-03-07T07:18:51.607 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-07T07:18:51.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-07T07:18:51.609 INFO:tasks.ceph.mon.c:Started 2024-03-07T07:18:51.609 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-07T07:18:51.609 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-07T07:18:51.609 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-07T07:18:51.611 INFO:tasks.ceph.mgr.y:Started 2024-03-07T07:18:51.611 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-07T07:18:51.611 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-07T07:18:51.613 INFO:tasks.ceph.mgr.x:Started 2024-03-07T07:18:51.613 INFO:tasks.ceph:Setting crush tunables to default 2024-03-07T07:18:51.614 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph osd crush tunables default 2024-03-07T07:18:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.715+0000 7f1962883640 1 Processor -- start 2024-03-07T07:18:51.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.715+0000 7f1962883640 1 -- start start 2024-03-07T07:18:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f1962883640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f1962883640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f1962883640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195c0cdd40 con 0x7f195c15beb0 2024-03-07T07:18:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f1962883640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c0cf210 con 0x7f195c06d8d0 2024-03-07T07:18:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f1962883640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c0d19d0 con 0x7f195c0d1020 2024-03-07T07:18:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f195b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f195bfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f195c15beb0 0x7f195c0ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44544/0 (socket says 172.21.15.64:44544) 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f195bfff640 1 -- 172.21.15.64:0/591680402 learned_addr learned my addr 172.21.15.64:0/591680402 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f195b7fe640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 msgr2=0x7f195c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f195b7fe640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 msgr2=0x7f195c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f195b7fe640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.716+0000 7f195b7fe640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.717+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.717+0000 7f1960e20640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 msgr2=0x7f195c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.717+0000 7f1960e20640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 msgr2=0x7f195c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.717+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T07:18:51.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.717+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T07:18:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.917+0000 7f195b7fe640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.917+0000 7f195b7fe640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 msgr2=0x7f195c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-07T07:18:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.917+0000 7f195b7fe640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 msgr2=0x7f195c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T07:18:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.917+0000 7f195b7fe640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T07:18:51.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.917+0000 7f195b7fe640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T07:18:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.918+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.918+0000 7f1960e20640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 msgr2=0x7f195c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-07T07:18:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.918+0000 7f1960e20640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 msgr2=0x7f195c0d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T07:18:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.918+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T07:18:51.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.918+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T07:18:51.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.921+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664104112 0 0) 0x7f195c0cdd40 con 0x7f195c15beb0 2024-03-07T07:18:51.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.921+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1944003520 con 0x7f195c15beb0 2024-03-07T07:18:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4168686684 0 0) 0x7f1944003520 con 0x7f195c15beb0 2024-03-07T07:18:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 msgr2=0x7f195c0d1490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T07:18:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.120:3300/0 conn(0x7f195c0d1020 0x7f195c0d1490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 msgr2=0x7f195c0b4370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T07:18:51.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 --2- 172.21.15.64:0/591680402 >> v2:172.21.15.64:3300/0 conn(0x7f195c06d8d0 0x7f195c0b4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0ce020 con 0x7f195c15beb0 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f194c001670 con 0x7f195c15beb0 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 >> v1:172.21.15.64:6789/0 conn(0x7f195c15beb0 legacy=0x7f195c0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19440039b0 0x7f1944003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1944004800 0x7f1944004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440051c0 0x7f194400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0cf210 con 0x7f19440051c0 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0d19d0 con 0x7f19440039b0 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194400c170 con 0x7f1944004800 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195bfff640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19440039b0 0x7f1944003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440051c0 0x7f194400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195b7fe640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1944004800 0x7f1944004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195bfff640 1 -- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1944004800 msgr2=0x7f1944004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195bfff640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1944004800 0x7f1944004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195bfff640 1 -- 172.21.15.64:0/591680402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440051c0 msgr2=0x7f194400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195bfff640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440051c0 0x7f194400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195bfff640 1 -- 172.21.15.64:0/591680402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944003520 con 0x7f19440039b0 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f195b7fe640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1944004800 0x7f1944004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f1960e20640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440051c0 0x7f194400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.922+0000 7f1960e20640 1 -- 172.21.15.64:0/591680402 reap_dead start 2024-03-07T07:18:51.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.923+0000 7f195bfff640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19440039b0 0x7f1944003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f194c007fd0 tx=0x7f1944005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6129261e24680a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:51.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.928+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c007d50 con 0x7f19440039b0 2024-03-07T07:18:51.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.928+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c030070 con 0x7f19440039b0 2024-03-07T07:18:51.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.928+0000 7f195affd640 1 -- 172.21.15.64:0/591680402 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c005d10 con 0x7f19440039b0 2024-03-07T07:18:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.929+0000 7f1962883640 1 -- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19440039b0 msgr2=0x7f1944003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.929+0000 7f1962883640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19440039b0 0x7f1944003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f194c007fd0 tx=0x7f1944005830 comp rx=0 tx=0).stop 2024-03-07T07:18:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.929+0000 7f1962883640 1 -- 172.21.15.64:0/591680402 shutdown_connections 2024-03-07T07:18:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.929+0000 7f1962883640 1 --2- 172.21.15.64:0/591680402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19440039b0 0x7f1944003dd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.929+0000 7f1962883640 1 -- 172.21.15.64:0/591680402 >> 172.21.15.64:0/591680402 conn(0x7f195c070400 msgr2=0x7f195c070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.929+0000 7f1962883640 1 -- 172.21.15.64:0/591680402 shutdown_connections 2024-03-07T07:18:51.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.929+0000 7f1962883640 1 -- 172.21.15.64:0/591680402 wait complete. 2024-03-07T07:18:51.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.931+0000 7f1962883640 1 Processor -- start 2024-03-07T07:18:51.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.931+0000 7f1962883640 1 -- start start 2024-03-07T07:18:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f1962883640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440039b0 0x7f195c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f1962883640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f195c0c82e0 0x7f195c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f1962883640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c14ebb0 0x7f195c14f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:51.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f1962883640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0ce5e0 con 0x7f19440039b0 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f1962883640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c06d0f0 con 0x7f195c0c82e0 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f1962883640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0cd9f0 con 0x7f195c14ebb0 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f195bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440039b0 0x7f195c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f195bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440039b0 0x7f195c15c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49640/0 (socket says 172.21.15.64:49640) 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f195bfff640 1 -- 172.21.15.64:0/2601080024 learned_addr learned my addr 172.21.15.64:0/2601080024 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.932+0000 7f195b7fe640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f195c0c82e0 0x7f195c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1960e20640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c14ebb0 0x7f195c14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f195bfff640 1 -- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c14ebb0 msgr2=0x7f195c14f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f195bfff640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c14ebb0 0x7f195c14f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f195bfff640 1 -- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f195c0c82e0 msgr2=0x7f195c0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f195bfff640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f195c0c82e0 0x7f195c0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f195bfff640 1 -- 172.21.15.64:0/2601080024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c15d5f0 con 0x7f19440039b0 2024-03-07T07:18:51.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f195bfff640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440039b0 0x7f195c15c420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f194c009000 tx=0x7f194c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1958ff9640 1 -- 172.21.15.64:0/2601080024 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c004bf0 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1958ff9640 1 -- 172.21.15.64:0/2601080024 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c004d90 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c15cd50 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1958ff9640 1 -- 172.21.15.64:0/2601080024 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c007270 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c15d310 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1958ff9640 1 -- 172.21.15.64:0/2601080024 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f194c035470 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.933+0000 7f1958ff9640 1 -- 172.21.15.64:0/2601080024 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f194c041850 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.934+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c062c90 con 0x7f19440039b0 2024-03-07T07:18:51.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:51.936+0000 7f1958ff9640 1 -- 172.21.15.64:0/2601080024 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f194c0358a0 con 0x7f19440039b0 2024-03-07T07:18:52.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.003+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f195c0b6b00 con 0x7f19440039b0 2024-03-07T07:18:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.005+0000 7f1958ff9640 1 -- 172.21.15.64:0/2601080024 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f194c041aa0 con 0x7f19440039b0 2024-03-07T07:18:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440039b0 msgr2=0x7f195c15c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440039b0 0x7f195c15c420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f194c009000 tx=0x7f194c000fe0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 shutdown_connections 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c14ebb0 0x7f195c14f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f195c0c82e0 0x7f195c0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 --2- 172.21.15.64:0/2601080024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19440039b0 0x7f195c15c420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 >> 172.21.15.64:0/2601080024 conn(0x7f195c070400 msgr2=0x7f195c072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 shutdown_connections 2024-03-07T07:18:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:52.006+0000 7f1962883640 1 -- 172.21.15.64:0/2601080024 wait complete. 2024-03-07T07:18:52.017 INFO:tasks.ceph:check_enable_crimson: False 2024-03-07T07:18:52.017 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-07T07:18:52.017 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:52.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-07T07:18:52.044 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:52.044 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-07T07:18:52.112 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:52.112 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-07T07:18:52.179 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:18:52.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-07T07:18:52.246 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:52.246 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-07T07:18:52.273 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:52.273 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-07T07:18:52.341 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:52.342 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-07T07:18:52.403 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:52.403+0000 7fe7e5c45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T07:18:52.405 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:52.405+0000 7f8828a3e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T07:18:52.410 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:18:52.410 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-07T07:18:52.459 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:52.459+0000 7fe7e5c45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T07:18:52.460 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:52.461+0000 7f8828a3e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T07:18:52.476 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 6778ad92-ea90-4f4e-800a-a1934ea4f756 0 2024-03-07T07:18:52.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.617+0000 7fcc097c9640 1 Processor -- start 2024-03-07T07:18:52.617 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.617+0000 7fcc097c9640 1 -- start start 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc097c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fcc040c5640 0x7fcc040b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc097c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040b44c0 con 0x7fcc04071b20 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc097c9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0415c350 con 0x7fcc04156310 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc097c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040c6cb0 con 0x7fcc040c5640 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc02ffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fcc04156310 0x7fcc04071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:60154/0 (socket says 172.21.15.120:60154) 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc02ffd640 1 -- 172.21.15.120:0/2911601848 learned_addr learned my addr 172.21.15.120:0/2911601848 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc037fe640 1 --2- 172.21.15.120:0/2911601848 >> v2:172.21.15.64:3300/0 conn(0x7fcc040c5640 0x7fcc040b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc01ffb640 1 -- 172.21.15.120:0/2911601848 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593468654 0 0) 0x7fcc0415c350 con 0x7fcc04156310 2024-03-07T07:18:52.619 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.618+0000 7fcc01ffb640 1 -- 172.21.15.120:0/2911601848 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf0003520 con 0x7fcc04156310 2024-03-07T07:18:52.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.619+0000 7fcc037fe640 1 -- 172.21.15.120:0/2911601848 >> v1:172.21.15.120:6789/0 conn(0x7fcc04156310 legacy=0x7fcc04071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.619+0000 7fcc037fe640 1 -- 172.21.15.120:0/2911601848 >> v1:172.21.15.64:6789/0 conn(0x7fcc04071b20 legacy=0x7fcc0415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.619+0000 7fcc01ffb640 1 -- 172.21.15.120:0/2911601848 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608163597 0 0) 0x7fcc040b44c0 con 0x7fcc04071b20 2024-03-07T07:18:52.620 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.619+0000 7fcc037fe640 1 -- 172.21.15.120:0/2911601848 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0415c5f0 con 0x7fcc040c5640 2024-03-07T07:18:52.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.619+0000 7fcc037fe640 1 --2- 172.21.15.120:0/2911601848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 0x7fcc040b2c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009d90 tx=0x7fcbfc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12f5baf539dc01c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:52.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.619+0000 7fcc01ffb640 1 -- 172.21.15.120:0/2911601848 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc039070 con 0x7fcc040c5640 2024-03-07T07:18:52.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc01ffb640 1 -- 172.21.15.120:0/2911601848 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc02cd00 con 0x7fcc040c5640 2024-03-07T07:18:52.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc01ffb640 1 -- 172.21.15.120:0/2911601848 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc034660 con 0x7fcc040c5640 2024-03-07T07:18:52.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc097c9640 1 -- 172.21.15.120:0/2911601848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 msgr2=0x7fcc040b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc097c9640 1 --2- 172.21.15.120:0/2911601848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 0x7fcc040b2c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009d90 tx=0x7fcbfc02bf50 comp rx=0 tx=0).stop 2024-03-07T07:18:52.621 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc097c9640 1 -- 172.21.15.120:0/2911601848 shutdown_connections 2024-03-07T07:18:52.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc097c9640 1 --2- 172.21.15.120:0/2911601848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 0x7fcc040b2c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc097c9640 1 -- 172.21.15.120:0/2911601848 >> 172.21.15.120:0/2911601848 conn(0x7fcc040727c0 msgr2=0x7fcc04072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:52.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc097c9640 1 -- 172.21.15.120:0/2911601848 shutdown_connections 2024-03-07T07:18:52.622 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.620+0000 7fcc097c9640 1 -- 172.21.15.120:0/2911601848 wait complete. 2024-03-07T07:18:52.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.621+0000 7fcc097c9640 1 Processor -- start 2024-03-07T07:18:52.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc097c9640 1 -- start start 2024-03-07T07:18:52.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc097c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04071b20 0x7fcc0415e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.624 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc097c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 0x7fcc0414bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc097c9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcc04156310 0x7fcc040cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc097c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415cbb0 con 0x7fcc040c5640 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc097c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040b3160 con 0x7fcc04071b20 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc097c9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415ce90 con 0x7fcc04156310 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04071b20 0x7fcc0415e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc037fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcc04156310 0x7fcc040cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc02ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04071b20 0x7fcc0415e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39644/0 (socket says 172.21.15.120:39644) 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc02ffd640 1 -- 172.21.15.120:0/322199048 learned_addr learned my addr 172.21.15.120:0/322199048 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.622+0000 7fcc027fc640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 0x7fcc0414bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc037fe640 1 -- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04071b20 msgr2=0x7fcc0415e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc037fe640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04071b20 0x7fcc0415e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc037fe640 1 -- 172.21.15.120:0/322199048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 msgr2=0x7fcc0414bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc037fe640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 0x7fcc0414bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.625 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc037fe640 1 -- 172.21.15.120:0/322199048 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040cdb00 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc02ffd640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04071b20 0x7fcc0415e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc037fe640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcc04156310 0x7fcc040cd120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc033000 tx=0x7fcbfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcbfb7fe640 1 -- 172.21.15.120:0/322199048 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc039050 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcbfb7fe640 1 -- 172.21.15.120:0/322199048 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc03a040 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0415d790 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcbfb7fe640 1 -- 172.21.15.120:0/322199048 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc009450 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0415dd50 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcbfb7fe640 1 -- 172.21.15.120:0/322199048 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcbfc047020 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.623+0000 7fcbfb7fe640 1 -- 172.21.15.120:0/322199048 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcbfc040570 con 0x7fcc04156310 2024-03-07T07:18:52.626 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.624+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc04062c90 con 0x7fcc04156310 2024-03-07T07:18:52.627 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.626+0000 7fcbfb7fe640 1 -- 172.21.15.120:0/322199048 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcbfc03e610 con 0x7fcc04156310 2024-03-07T07:18:52.698 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.698+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6778ad92-ea90-4f4e-800a-a1934ea4f756", "id": 0} v 0) v1 -- 0x7fcc040ccf00 con 0x7fcc04156310 2024-03-07T07:18:52.701 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.701+0000 7fcbfb7fe640 1 -- 172.21.15.120:0/322199048 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6778ad92-ea90-4f4e-800a-a1934ea4f756", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcbfc03e7f0 con 0x7fcc04156310 2024-03-07T07:18:52.701 INFO:teuthology.orchestra.run.smithi120.stdout:0 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcc04156310 msgr2=0x7fcc040cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcc04156310 0x7fcc040cd120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc033000 tx=0x7fcbfc004a40 comp rx=0 tx=0).stop 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 shutdown_connections 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc040c5640 0x7fcc0414bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcc04156310 0x7fcc040cd120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 --2- 172.21.15.120:0/322199048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcc04071b20 0x7fcc0415e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 >> 172.21.15.120:0/322199048 conn(0x7fcc040727c0 msgr2=0x7fcc04070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 shutdown_connections 2024-03-07T07:18:52.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.702+0000 7fcc097c9640 1 -- 172.21.15.120:0/322199048 wait complete. 2024-03-07T07:18:52.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new b56d13e1-6c08-4768-8b35-cbaa4ca12a0f 1 2024-03-07T07:18:52.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.812+0000 7fd2fc2d0640 1 Processor -- start 2024-03-07T07:18:52.812 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.812+0000 7fd2fc2d0640 1 -- start start 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fc2d0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd2f4145780 0x7fd2f40b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fc2d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd2f40b0230 0x7fd2f40b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fc2d0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f40b2f70 0x7fd2f4070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fc2d0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f40b33e0 con 0x7fd2f40b0230 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fc2d0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f40b3e40 con 0x7fd2f4145780 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fc2d0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4145b60 con 0x7fd2f40b2f70 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fa86d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f40b2f70 0x7fd2f4070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fa06c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd2f4145780 0x7fd2f40b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fa86d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f40b2f70 0x7fd2f4070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35736/0 (socket says 172.21.15.120:35736) 2024-03-07T07:18:52.814 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fa06c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd2f4145780 0x7fd2f40b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39652/0 (socket says 172.21.15.120:39652) 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2fa86d640 1 -- 172.21.15.120:0/2034898165 learned_addr learned my addr 172.21.15.120:0/2034898165 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.813+0000 7fd2f986b640 1 --2- 172.21.15.120:0/2034898165 >> v2:172.21.15.64:3300/0 conn(0x7fd2f40b0230 0x7fd2f40b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2fa86d640 1 -- 172.21.15.120:0/2034898165 >> v2:172.21.15.120:3300/0 conn(0x7fd2f4145780 msgr2=0x7fd2f40b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2fa86d640 1 --2- 172.21.15.120:0/2034898165 >> v2:172.21.15.120:3300/0 conn(0x7fd2f4145780 0x7fd2f40b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2fa86d640 1 -- 172.21.15.120:0/2034898165 >> v2:172.21.15.64:3300/0 conn(0x7fd2f40b0230 msgr2=0x7fd2f40b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2fa86d640 1 --2- 172.21.15.120:0/2034898165 >> v2:172.21.15.64:3300/0 conn(0x7fd2f40b0230 0x7fd2f40b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2fa86d640 1 -- 172.21.15.120:0/2034898165 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40b40b0 con 0x7fd2f40b2f70 2024-03-07T07:18:52.815 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2fa06c640 1 --2- 172.21.15.120:0/2034898165 >> v2:172.21.15.120:3300/0 conn(0x7fd2f4145780 0x7fd2f40b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2fa86d640 1 --2- 172.21.15.120:0/2034898165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f40b2f70 0x7fd2f4070440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b6270 tx=0x7fd2ec02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4da5f61a5f1d118 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:52.816 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2f906a640 1 -- 172.21.15.120:0/2034898165 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec02ca70 con 0x7fd2f40b2f70 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2f906a640 1 -- 172.21.15.120:0/2034898165 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec02cc10 con 0x7fd2f40b2f70 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.814+0000 7fd2f906a640 1 -- 172.21.15.120:0/2034898165 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0343f0 con 0x7fd2f40b2f70 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/2034898165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f40b2f70 msgr2=0x7fd2f4070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/2034898165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f40b2f70 0x7fd2f4070440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b6270 tx=0x7fd2ec02bed0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/2034898165 shutdown_connections 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/2034898165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f40b2f70 0x7fd2f4070440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/2034898165 >> v2:172.21.15.64:3300/0 conn(0x7fd2f40b0230 0x7fd2f40b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/2034898165 >> v2:172.21.15.120:3300/0 conn(0x7fd2f4145780 0x7fd2f40b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/2034898165 >> 172.21.15.120:0/2034898165 conn(0x7fd2f406b040 msgr2=0x7fd2f406b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/2034898165 shutdown_connections 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.815+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/2034898165 wait complete. 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.816+0000 7fd2fc2d0640 1 Processor -- start 2024-03-07T07:18:52.817 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.816+0000 7fd2fc2d0640 1 -- start start 2024-03-07T07:18:52.819 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fc2d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2f40b0230 0x7fd2f4073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fc2d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2f40b2f70 0x7fd2f409d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fc2d0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f4145780 0x7fd2f415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fc2d0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40b4950 con 0x7fd2f40b2f70 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fc2d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40b4670 con 0x7fd2f40b0230 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fc2d0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40b4e70 con 0x7fd2f4145780 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2f40b0230 0x7fd2f4073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2f40b0230 0x7fd2f4073080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39666/0 (socket says 172.21.15.120:39666) 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 -- 172.21.15.120:0/3910310630 learned_addr learned my addr 172.21.15.120:0/3910310630 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa86d640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f4145780 0x7fd2f415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2f986b640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2f40b2f70 0x7fd2f409d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 -- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f4145780 msgr2=0x7fd2f415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f4145780 0x7fd2f415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.820 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 -- 172.21.15.120:0/3910310630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2f40b2f70 msgr2=0x7fd2f409d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2f40b2f70 0x7fd2f409d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa06c640 1 -- 172.21.15.120:0/3910310630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40ce490 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2fa86d640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f4145780 0x7fd2f415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.817+0000 7fd2f986b640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2f40b2f70 0x7fd2f409d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2fa06c640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2f40b0230 0x7fd2f4073080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e990 tx=0x7fd2f000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2eaffd640 1 -- 172.21.15.120:0/3910310630 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0017070 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2eaffd640 1 -- 172.21.15.120:0/3910310630 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f000c9e0 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2eaffd640 1 -- 172.21.15.120:0/3910310630 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000cdb0 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f4071a80 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f4072040 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2eaffd640 1 -- 172.21.15.120:0/3910310630 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd2f001a070 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.818+0000 7fd2eaffd640 1 -- 172.21.15.120:0/3910310630 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fd2f0013470 con 0x7fd2f40b0230 2024-03-07T07:18:52.821 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.819+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f4062c90 con 0x7fd2f40b0230 2024-03-07T07:18:52.822 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.821+0000 7fd2eaffd640 1 -- 172.21.15.120:0/3910310630 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd2f40ce490 con 0x7fd2f40b0230 2024-03-07T07:18:52.891 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.891+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b56d13e1-6c08-4768-8b35-cbaa4ca12a0f", "id": 1} v 0) v1 -- 0x7fd2f40d12b0 con 0x7fd2f40b0230 2024-03-07T07:18:52.894 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.894+0000 7fd2eaffd640 1 -- 172.21.15.120:0/3910310630 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b56d13e1-6c08-4768-8b35-cbaa4ca12a0f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd2f0006020 con 0x7fd2f40b0230 2024-03-07T07:18:52.895 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2f40b0230 msgr2=0x7fd2f4073080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2f40b0230 0x7fd2f4073080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e990 tx=0x7fd2f000ee50 comp rx=0 tx=0).stop 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 shutdown_connections 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2f40b2f70 0x7fd2f409d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2f4145780 0x7fd2f415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 --2- 172.21.15.120:0/3910310630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2f40b0230 0x7fd2f4073080 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 >> 172.21.15.120:0/3910310630 conn(0x7fd2f406b040 msgr2=0x7fd2f41418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 shutdown_connections 2024-03-07T07:18:52.896 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:52.896+0000 7fd2fc2d0640 1 -- 172.21.15.120:0/3910310630 wait complete. 2024-03-07T07:18:52.909 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 7fe794a2-99e7-4e41-aacc-573fee75b2ab 2 2024-03-07T07:18:53.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.009+0000 7f5842bfc640 1 Processor -- start 2024-03-07T07:18:53.009 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.009+0000 7f5842bfc640 1 -- start start 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.009+0000 7f5842bfc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f583c161830 0x7f583c0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5842bfc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c06f1e0 con 0x7f583c071fa0 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5842bfc640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c0ae6a0 con 0x7f583c0afe70 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5842bfc640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c161c10 con 0x7f583c161830 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5840998640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f583c161830 0x7f583c0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5840998640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f583c161830 0x7f583c0b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39680/0 (socket says 172.21.15.120:39680) 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5840998640 1 -- 172.21.15.120:0/1079794635 learned_addr learned my addr 172.21.15.120:0/1079794635 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5841199640 1 --1- 172.21.15.120:0/1079794635 >> v1:172.21.15.120:6789/0 conn(0x7f583c0afe70 0x7f583c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:60166/0 (socket says 172.21.15.120:60166) 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5840998640 1 -- 172.21.15.120:0/1079794635 >> v1:172.21.15.120:6789/0 conn(0x7f583c0afe70 legacy=0x7f583c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5840998640 1 -- 172.21.15.120:0/1079794635 >> v1:172.21.15.64:6789/0 conn(0x7f583c071fa0 legacy=0x7f583c06e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.011 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.010+0000 7f5840998640 1 -- 172.21.15.120:0/1079794635 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c071af0 con 0x7f583c161830 2024-03-07T07:18:53.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5840998640 1 --2- 172.21.15.120:0/1079794635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c161830 0x7f583c0b2bd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5830009870 tx=0x7f583002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78fc591d1a8d5dba server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f583b7fe640 1 -- 172.21.15.120:0/1079794635 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830034070 con 0x7f583c161830 2024-03-07T07:18:53.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f583b7fe640 1 -- 172.21.15.120:0/1079794635 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830030860 con 0x7f583c161830 2024-03-07T07:18:53.012 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f583b7fe640 1 -- 172.21.15.120:0/1079794635 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830030c30 con 0x7f583c161830 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5842bfc640 1 -- 172.21.15.120:0/1079794635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c161830 msgr2=0x7f583c0b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5842bfc640 1 --2- 172.21.15.120:0/1079794635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c161830 0x7f583c0b2bd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5830009870 tx=0x7f583002b9a0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5842bfc640 1 -- 172.21.15.120:0/1079794635 shutdown_connections 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5842bfc640 1 --2- 172.21.15.120:0/1079794635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c161830 0x7f583c0b2bd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5842bfc640 1 -- 172.21.15.120:0/1079794635 >> 172.21.15.120:0/1079794635 conn(0x7f583c0b6460 msgr2=0x7f583c0b6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5842bfc640 1 -- 172.21.15.120:0/1079794635 shutdown_connections 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.011+0000 7f5842bfc640 1 -- 172.21.15.120:0/1079794635 wait complete. 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 Processor -- start 2024-03-07T07:18:53.013 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 -- start start 2024-03-07T07:18:53.015 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c071fa0 0x7f583c150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f583c0afe70 0x7f583c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f583c161830 0x7f583c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0ae940 con 0x7f583c161830 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0b4430 con 0x7f583c071fa0 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.013+0000 7f5842bfc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0b4710 con 0x7f583c0afe70 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c071fa0 0x7f583c150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f583bfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f583c0afe70 0x7f583c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f583bfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f583c0afe70 0x7f583c157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35742/0 (socket says 172.21.15.120:35742) 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c071fa0 0x7f583c150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39682/0 (socket says 172.21.15.120:39682) 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f583bfff640 1 -- 172.21.15.120:0/560008147 learned_addr learned my addr 172.21.15.120:0/560008147 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5841199640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f583c161830 0x7f583c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.016 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 -- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f583c0afe70 msgr2=0x7f583c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f583c0afe70 0x7f583c157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 -- 172.21.15.120:0/560008147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f583c161830 msgr2=0x7f583c1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f583c161830 0x7f583c1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 -- 172.21.15.120:0/560008147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c15a370 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f583bfff640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f583c0afe70 0x7f583c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5841199640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f583c161830 0x7f583c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5840998640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c071fa0 0x7f583c150850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5830033000 tx=0x7f58300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f58397fa640 1 -- 172.21.15.120:0/560008147 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830034040 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f58397fa640 1 -- 172.21.15.120:0/560008147 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58300433f0 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f58397fa640 1 -- 172.21.15.120:0/560008147 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300437e0 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c15a650 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c15ac10 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.014+0000 7f58397fa640 1 -- 172.21.15.120:0/560008147 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f583004a060 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.015+0000 7f58397fa640 1 -- 172.21.15.120:0/560008147 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f583002f7d0 con 0x7f583c071fa0 2024-03-07T07:18:53.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.015+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c062c90 con 0x7f583c071fa0 2024-03-07T07:18:53.019 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.019+0000 7f58397fa640 1 -- 172.21.15.120:0/560008147 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f583003e070 con 0x7f583c071fa0 2024-03-07T07:18:53.024 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.026+0000 7f8828a3e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T07:18:53.036 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.036+0000 7fe7e5c45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T07:18:53.079 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.079+0000 7f8828a3e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T07:18:53.090 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.090+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7fe794a2-99e7-4e41-aacc-573fee75b2ab", "id": 2} v 0) v1 -- 0x7f583c0af4c0 con 0x7f583c071fa0 2024-03-07T07:18:53.092 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.092+0000 7fe7e5c45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T07:18:53.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.093+0000 7f58397fa640 1 -- 172.21.15.120:0/560008147 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7fe794a2-99e7-4e41-aacc-573fee75b2ab", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5830030720 con 0x7f583c071fa0 2024-03-07T07:18:53.093 INFO:teuthology.orchestra.run.smithi120.stdout:2 2024-03-07T07:18:53.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c071fa0 msgr2=0x7f583c150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c071fa0 0x7f583c150850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5830033000 tx=0x7f58300046c0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 shutdown_connections 2024-03-07T07:18:53.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f583c161830 0x7f583c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.094 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f583c0afe70 0x7f583c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 --2- 172.21.15.120:0/560008147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f583c071fa0 0x7f583c150850 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 >> 172.21.15.120:0/560008147 conn(0x7f583c0b6460 msgr2=0x7f583c15f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 shutdown_connections 2024-03-07T07:18:53.095 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.094+0000 7f5842bfc640 1 -- 172.21.15.120:0/560008147 wait complete. 2024-03-07T07:18:53.104 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 49430b37-e1cf-41c4-89d9-d463a63b8412 3 2024-03-07T07:18:53.127 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.127+0000 7f8828a3e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T07:18:53.139 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.139+0000 7fe7e5c45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T07:18:53.183 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.184+0000 7f8828a3e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T07:18:53.195 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.195+0000 7fe7e5c45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T07:18:53.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.204+0000 7f3e9a314640 1 Processor -- start 2024-03-07T07:18:53.204 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.204+0000 7f3e9a314640 1 -- start start 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e9a314640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940cef50 0x7f3e940cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e9a314640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3e940aee50 0x7f3e940af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e9a314640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3e9415c530 0x7f3e9415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e9a314640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9415b840 con 0x7f3e940aee50 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e9a314640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e940cf870 con 0x7f3e9415c530 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e9a314640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9406eec0 con 0x7f3e940cef50 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940cef50 0x7f3e940cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e988b1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3e9415c530 0x7f3e9415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940cef50 0x7f3e940cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35758/0 (socket says 172.21.15.120:35758) 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 -- 172.21.15.120:0/2082330867 learned_addr learned my addr 172.21.15.120:0/2082330867 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.206 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e937fe640 1 --2- 172.21.15.120:0/2082330867 >> v2:172.21.15.64:3300/0 conn(0x7f3e940aee50 0x7f3e940af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 -- 172.21.15.120:0/2082330867 >> v2:172.21.15.120:3300/0 conn(0x7f3e9415c530 msgr2=0x7f3e9415c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 --2- 172.21.15.120:0/2082330867 >> v2:172.21.15.120:3300/0 conn(0x7f3e9415c530 0x7f3e9415c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 -- 172.21.15.120:0/2082330867 >> v2:172.21.15.64:3300/0 conn(0x7f3e940aee50 msgr2=0x7f3e940af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 --2- 172.21.15.120:0/2082330867 >> v2:172.21.15.64:3300/0 conn(0x7f3e940aee50 0x7f3e940af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e93fff640 1 -- 172.21.15.120:0/2082330867 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9406f1a0 con 0x7f3e940cef50 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.205+0000 7f3e988b1640 1 --2- 172.21.15.120:0/2082330867 >> v2:172.21.15.120:3300/0 conn(0x7f3e9415c530 0x7f3e9415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.206+0000 7f3e93fff640 1 --2- 172.21.15.120:0/2082330867 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940cef50 0x7f3e940cf330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e9415e9d0 tx=0x7f3e8402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cb35e0c152278b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.206+0000 7f3e92ffd640 1 -- 172.21.15.120:0/2082330867 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84035020 con 0x7f3e940cef50 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.206+0000 7f3e92ffd640 1 -- 172.21.15.120:0/2082330867 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84031870 con 0x7f3e940cef50 2024-03-07T07:18:53.207 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.206+0000 7f3e92ffd640 1 -- 172.21.15.120:0/2082330867 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84031c40 con 0x7f3e940cef50 2024-03-07T07:18:53.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 -- 172.21.15.120:0/2082330867 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940cef50 msgr2=0x7f3e940cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 --2- 172.21.15.120:0/2082330867 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940cef50 0x7f3e940cf330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e9415e9d0 tx=0x7f3e8402b910 comp rx=0 tx=0).stop 2024-03-07T07:18:53.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 -- 172.21.15.120:0/2082330867 shutdown_connections 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 --2- 172.21.15.120:0/2082330867 >> v2:172.21.15.120:3300/0 conn(0x7f3e9415c530 0x7f3e9415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 --2- 172.21.15.120:0/2082330867 >> v2:172.21.15.64:3300/0 conn(0x7f3e940aee50 0x7f3e940af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 --2- 172.21.15.120:0/2082330867 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940cef50 0x7f3e940cf330 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 -- 172.21.15.120:0/2082330867 >> 172.21.15.120:0/2082330867 conn(0x7f3e940cd710 msgr2=0x7f3e940cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 -- 172.21.15.120:0/2082330867 shutdown_connections 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.207+0000 7f3e9a314640 1 -- 172.21.15.120:0/2082330867 wait complete. 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.208+0000 7f3e9a314640 1 Processor -- start 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.208+0000 7f3e9a314640 1 -- start start 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940aee50 0x7f3e941f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e940cef50 0x7f3e941f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e9415c530 0x7f3e941f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940ae340 con 0x7f3e940cef50 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940ae930 con 0x7f3e9415c530 2024-03-07T07:18:53.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940b2670 con 0x7f3e940aee50 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940aee50 0x7f3e941f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940aee50 0x7f3e941f0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35764/0 (socket says 172.21.15.120:35764) 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 -- 172.21.15.120:0/258008704 learned_addr learned my addr 172.21.15.120:0/258008704 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e988b1640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e9415c530 0x7f3e941f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e937fe640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e940cef50 0x7f3e941f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 -- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e9415c530 msgr2=0x7f3e941f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e9415c530 0x7f3e941f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 -- 172.21.15.120:0/258008704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e940cef50 msgr2=0x7f3e941f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e940cef50 0x7f3e941f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 -- 172.21.15.120:0/258008704 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94156690 con 0x7f3e940aee50 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e988b1640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e9415c530 0x7f3e941f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e93fff640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940aee50 0x7f3e941f0f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e8402b950 tx=0x7f3e840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.212 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e90ff9640 1 -- 172.21.15.120:0/258008704 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84040070 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e90ff9640 1 -- 172.21.15.120:0/258008704 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84031e10 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9414fbf0 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e90ff9640 1 -- 172.21.15.120:0/258008704 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84035040 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.209+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e941501b0 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.210+0000 7f3e90ff9640 1 -- 172.21.15.120:0/258008704 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3e8404b060 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.210+0000 7f3e90ff9640 1 -- 172.21.15.120:0/258008704 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f3e8403a8b0 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.211+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e94062c90 con 0x7f3e940aee50 2024-03-07T07:18:53.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.213+0000 7f3e90ff9640 1 -- 172.21.15.120:0/258008704 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3e8403a420 con 0x7f3e940aee50 2024-03-07T07:18:53.229 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.229+0000 7f8828a3e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T07:18:53.239 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.240+0000 7fe7e5c45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T07:18:53.284 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.285+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "49430b37-e1cf-41c4-89d9-d463a63b8412", "id": 3} v 0) v1 -- 0x7f3e940d04f0 con 0x7f3e940aee50 2024-03-07T07:18:53.287 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.288+0000 7f3e90ff9640 1 -- 172.21.15.120:0/258008704 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49430b37-e1cf-41c4-89d9-d463a63b8412", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3e8403ab50 con 0x7f3e940aee50 2024-03-07T07:18:53.287 INFO:teuthology.orchestra.run.smithi120.stdout:3 2024-03-07T07:18:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940aee50 msgr2=0x7f3e941f0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940aee50 0x7f3e941f0f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e8402b950 tx=0x7f3e840046c0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 shutdown_connections 2024-03-07T07:18:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e940cef50 0x7f3e941f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e9415c530 0x7f3e941f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 --2- 172.21.15.120:0/258008704 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e940aee50 0x7f3e941f0f30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 >> 172.21.15.120:0/258008704 conn(0x7f3e940cd710 msgr2=0x7f3e94158780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 shutdown_connections 2024-03-07T07:18:53.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.290+0000 7f3e9a314640 1 -- 172.21.15.120:0/258008704 wait complete. 2024-03-07T07:18:53.303 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 3df056f1-81c0-4587-9b2b-66de4f62d0f2 4 2024-03-07T07:18:53.353 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.354+0000 7f8828a3e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T07:18:53.360 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.360+0000 7fe7e5c45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T07:18:53.426 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.427+0000 7f8828a3e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T07:18:53.433 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.433+0000 7fe7e5c45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T07:18:53.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.443+0000 7fc26929c640 1 Processor -- start 2024-03-07T07:18:53.442 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.443+0000 7fc26929c640 1 -- start start 2024-03-07T07:18:53.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc26929c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc26415bee0 0x7fc26406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc26929c640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc264071640 con 0x7fc2640b5b50 2024-03-07T07:18:53.443 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc26929c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2640cd6c0 con 0x7fc264070e10 2024-03-07T07:18:53.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc26929c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2640ceb90 con 0x7fc26415bee0 2024-03-07T07:18:53.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc262ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc26415bee0 0x7fc26406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc262ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc26415bee0 0x7fc26406f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39724/0 (socket says 172.21.15.120:39724) 2024-03-07T07:18:53.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc262ffd640 1 -- 172.21.15.120:0/2997626485 learned_addr learned my addr 172.21.15.120:0/2997626485 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc262ffd640 1 -- 172.21.15.120:0/2997626485 >> v1:172.21.15.120:6789/0 conn(0x7fc264070e10 legacy=0x7fc2640ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc262ffd640 1 -- 172.21.15.120:0/2997626485 >> v1:172.21.15.64:6789/0 conn(0x7fc2640b5b50 legacy=0x7fc2640b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.444 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.444+0000 7fc262ffd640 1 -- 172.21.15.120:0/2997626485 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2640cd9a0 con 0x7fc26415bee0 2024-03-07T07:18:53.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.445+0000 7fc262ffd640 1 --2- 172.21.15.120:0/2997626485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc26415bee0 0x7fc26406f0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc2640ced10 tx=0x7fc25802ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4f77c363447863 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.445+0000 7fc261ffb640 1 -- 172.21.15.120:0/2997626485 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25802f840 con 0x7fc26415bee0 2024-03-07T07:18:53.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.445+0000 7fc261ffb640 1 -- 172.21.15.120:0/2997626485 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25802f9e0 con 0x7fc26415bee0 2024-03-07T07:18:53.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.445+0000 7fc261ffb640 1 -- 172.21.15.120:0/2997626485 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25802fdb0 con 0x7fc26415bee0 2024-03-07T07:18:53.445 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.446+0000 7fc26929c640 1 -- 172.21.15.120:0/2997626485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc26415bee0 msgr2=0x7fc26406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.446+0000 7fc26929c640 1 --2- 172.21.15.120:0/2997626485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc26415bee0 0x7fc26406f0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc2640ced10 tx=0x7fc25802ba20 comp rx=0 tx=0).stop 2024-03-07T07:18:53.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.446+0000 7fc26929c640 1 -- 172.21.15.120:0/2997626485 shutdown_connections 2024-03-07T07:18:53.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.446+0000 7fc26929c640 1 --2- 172.21.15.120:0/2997626485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc26415bee0 0x7fc26406f0b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.446+0000 7fc26929c640 1 -- 172.21.15.120:0/2997626485 >> 172.21.15.120:0/2997626485 conn(0x7fc26406b370 msgr2=0x7fc26406b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.446+0000 7fc26929c640 1 -- 172.21.15.120:0/2997626485 shutdown_connections 2024-03-07T07:18:53.446 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.446+0000 7fc26929c640 1 -- 172.21.15.120:0/2997626485 wait complete. 2024-03-07T07:18:53.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.447+0000 7fc26929c640 1 Processor -- start 2024-03-07T07:18:53.448 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.447+0000 7fc26929c640 1 -- start start 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc26929c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc264070e10 0x7fc26413af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc26929c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2640b5b50 0x7fc26414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc26929c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26415bee0 0x7fc26414df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc26929c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640712a0 con 0x7fc26415bee0 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc26929c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640cdf60 con 0x7fc2640b5b50 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc26929c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc264071420 con 0x7fc264070e10 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc264070e10 0x7fc26413af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc2627fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2640b5b50 0x7fc26414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc264070e10 0x7fc26413af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35770/0 (socket says 172.21.15.120:35770) 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 -- 172.21.15.120:0/4142720981 learned_addr learned my addr 172.21.15.120:0/4142720981 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc2637fe640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26415bee0 0x7fc26414df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 -- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2640b5b50 msgr2=0x7fc26414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2640b5b50 0x7fc26414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 -- 172.21.15.120:0/4142720981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26415bee0 msgr2=0x7fc26414df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.449 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26415bee0 0x7fc26414df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 -- 172.21.15.120:0/4142720981 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26413b660 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc2627fc640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2640b5b50 0x7fc26414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc262ffd640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc264070e10 0x7fc26413af60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc258000c00 tx=0x7fc258004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc24b7fe640 1 -- 172.21.15.120:0/4142720981 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258047070 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.448+0000 7fc24b7fe640 1 -- 172.21.15.120:0/4142720981 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258042480 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.449+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26413b940 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.449+0000 7fc24b7fe640 1 -- 172.21.15.120:0/4142720981 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258042870 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.449+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26413bf00 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.449+0000 7fc24b7fe640 1 -- 172.21.15.120:0/4142720981 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc25804a070 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.449+0000 7fc24b7fe640 1 -- 172.21.15.120:0/4142720981 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fc25802d8e0 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.449+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc264062c90 con 0x7fc264070e10 2024-03-07T07:18:53.450 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.451+0000 7fc24b7fe640 1 -- 172.21.15.120:0/4142720981 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc258051070 con 0x7fc264070e10 2024-03-07T07:18:53.474 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.475+0000 7f8828a3e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T07:18:53.481 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.481+0000 7fe7e5c45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T07:18:53.521 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.522+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3df056f1-81c0-4587-9b2b-66de4f62d0f2", "id": 4} v 0) v1 -- 0x7fc2640cd290 con 0x7fc264070e10 2024-03-07T07:18:53.526 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.528+0000 7fc24b7fe640 1 -- 172.21.15.120:0/4142720981 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3df056f1-81c0-4587-9b2b-66de4f62d0f2", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc258033030 con 0x7fc264070e10 2024-03-07T07:18:53.527 INFO:teuthology.orchestra.run.smithi120.stdout:4 2024-03-07T07:18:53.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.528+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc264070e10 msgr2=0x7fc26413af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.528+0000 7fc26929c640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc264070e10 0x7fc26413af60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc258000c00 tx=0x7fc258004730 comp rx=0 tx=0).stop 2024-03-07T07:18:53.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.528+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 shutdown_connections 2024-03-07T07:18:53.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.528+0000 7fc26929c640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26415bee0 0x7fc26414df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.528+0000 7fc26929c640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2640b5b50 0x7fc26414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.528+0000 7fc26929c640 1 --2- 172.21.15.120:0/4142720981 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc264070e10 0x7fc26413af60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.528 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.529+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 >> 172.21.15.120:0/4142720981 conn(0x7fc26406b370 msgr2=0x7fc26415e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.529+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 shutdown_connections 2024-03-07T07:18:53.529 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.529+0000 7fc26929c640 1 -- 172.21.15.120:0/4142720981 wait complete. 2024-03-07T07:18:53.529 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.529+0000 7f8828a3e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T07:18:53.536 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.536+0000 7fe7e5c45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T07:18:53.538 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 87eae5ba-3309-4484-ad81-48d1bcbe2dde 5 2024-03-07T07:18:53.576 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.577+0000 7f8828a3e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T07:18:53.585 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.585+0000 7fe7e5c45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T07:18:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.676+0000 7f9492a5e640 1 Processor -- start 2024-03-07T07:18:53.676 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9492a5e640 1 -- start start 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9492a5e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f948c15e1d0 0x7f948c162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9492a5e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c162580 0x7f948c0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9492a5e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c0715b0 con 0x7f948c070e10 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9492a5e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0d2220 con 0x7f948c15e1d0 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9492a5e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0d2500 con 0x7f948c162580 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f948bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f948c15e1d0 0x7f948c162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9490ffb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c162580 0x7f948c0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f948bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f948c15e1d0 0x7f948c162a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39738/0 (socket says 172.21.15.120:39738) 2024-03-07T07:18:53.677 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f9490ffb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c162580 0x7f948c0d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35782/0 (socket says 172.21.15.120:35782) 2024-03-07T07:18:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.677+0000 7f948bfff640 1 -- 172.21.15.120:0/2214013292 learned_addr learned my addr 172.21.15.120:0/2214013292 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.678+0000 7f9490ffb640 1 -- 172.21.15.120:0/2214013292 >> v2:172.21.15.120:3300/0 conn(0x7f948c15e1d0 msgr2=0x7f948c162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.678+0000 7f9490ffb640 1 --2- 172.21.15.120:0/2214013292 >> v2:172.21.15.120:3300/0 conn(0x7f948c15e1d0 0x7f948c162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.678+0000 7f9490ffb640 1 -- 172.21.15.120:0/2214013292 >> v1:172.21.15.64:6789/0 conn(0x7f948c070e10 legacy=0x7f948c161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.678+0000 7f9490ffb640 1 -- 172.21.15.120:0/2214013292 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c071890 con 0x7f948c162580 2024-03-07T07:18:53.678 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.678+0000 7f948bfff640 1 --2- 172.21.15.120:0/2214013292 >> v2:172.21.15.120:3300/0 conn(0x7f948c15e1d0 0x7f948c162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:53.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.678+0000 7f9490ffb640 1 --2- 172.21.15.120:0/2214013292 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c162580 0x7f948c0d2990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9484009d90 tx=0x7f948402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=472071eda9aec6b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f948affd640 1 -- 172.21.15.120:0/2214013292 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484039070 con 0x7f948c162580 2024-03-07T07:18:53.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f948affd640 1 -- 172.21.15.120:0/2214013292 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948402cbc0 con 0x7f948c162580 2024-03-07T07:18:53.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f948affd640 1 -- 172.21.15.120:0/2214013292 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484034440 con 0x7f948c162580 2024-03-07T07:18:53.679 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 -- 172.21.15.120:0/2214013292 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c162580 msgr2=0x7f948c0d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 --2- 172.21.15.120:0/2214013292 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c162580 0x7f948c0d2990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9484009d90 tx=0x7f948402be10 comp rx=0 tx=0).stop 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 -- 172.21.15.120:0/2214013292 shutdown_connections 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 --2- 172.21.15.120:0/2214013292 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c162580 0x7f948c0d2990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 --2- 172.21.15.120:0/2214013292 >> v2:172.21.15.120:3300/0 conn(0x7f948c15e1d0 0x7f948c162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 -- 172.21.15.120:0/2214013292 >> 172.21.15.120:0/2214013292 conn(0x7f948c0d4230 msgr2=0x7f948c0d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 -- 172.21.15.120:0/2214013292 shutdown_connections 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.679+0000 7f9492a5e640 1 -- 172.21.15.120:0/2214013292 wait complete. 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.680+0000 7f9492a5e640 1 Processor -- start 2024-03-07T07:18:53.680 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9492a5e640 1 -- start start 2024-03-07T07:18:53.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9492a5e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f948c070e10 0x7f948c0c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9492a5e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c15e1d0 0x7f948c161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.682 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9492a5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c162580 0x7f948c149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9492a5e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c15e5b0 con 0x7f948c162580 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9492a5e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0712a0 con 0x7f948c070e10 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9492a5e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c15e730 con 0x7f948c15e1d0 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f948bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f948c070e10 0x7f948c0c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f948b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c15e1d0 0x7f948c161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f948b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c15e1d0 0x7f948c161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35784/0 (socket says 172.21.15.120:35784) 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f948b7fe640 1 -- 172.21.15.120:0/951961637 learned_addr learned my addr 172.21.15.120:0/951961637 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f948bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f948c070e10 0x7f948c0c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39746/0 (socket says 172.21.15.120:39746) 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.681+0000 7f9490ffb640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c162580 0x7f948c149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f948b7fe640 1 -- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f948c070e10 msgr2=0x7f948c0c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f948b7fe640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f948c070e10 0x7f948c0c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f948b7fe640 1 -- 172.21.15.120:0/951961637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c162580 msgr2=0x7f948c149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f948b7fe640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c162580 0x7f948c149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f948b7fe640 1 -- 172.21.15.120:0/951961637 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0ca4f0 con 0x7f948c15e1d0 2024-03-07T07:18:53.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f948bfff640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f948c070e10 0x7f948c0c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f948b7fe640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c15e1d0 0x7f948c161000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f948000c430 tx=0x7f948000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f9488ff9640 1 -- 172.21.15.120:0/951961637 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480017070 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f9488ff9640 1 -- 172.21.15.120:0/951961637 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94800052d0 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f9488ff9640 1 -- 172.21.15.120:0/951961637 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480003850 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c0c9450 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c0c9a10 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f9488ff9640 1 -- 172.21.15.120:0/951961637 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f948001a070 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.682+0000 7f9488ff9640 1 -- 172.21.15.120:0/951961637 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9480011c50 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.683+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c062c90 con 0x7f948c15e1d0 2024-03-07T07:18:53.684 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.685+0000 7f9488ff9640 1 -- 172.21.15.120:0/951961637 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9480021020 con 0x7f948c15e1d0 2024-03-07T07:18:53.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.756+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "87eae5ba-3309-4484-ad81-48d1bcbe2dde", "id": 5} v 0) v1 -- 0x7f948c0cf620 con 0x7f948c15e1d0 2024-03-07T07:18:53.760 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.761+0000 7f9488ff9640 1 -- 172.21.15.120:0/951961637 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87eae5ba-3309-4484-ad81-48d1bcbe2dde", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f948c0ca4f0 con 0x7f948c15e1d0 2024-03-07T07:18:53.760 INFO:teuthology.orchestra.run.smithi120.stdout:5 2024-03-07T07:18:53.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c15e1d0 msgr2=0x7f948c161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c15e1d0 0x7f948c161000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f948000c430 tx=0x7f948000c8f0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 shutdown_connections 2024-03-07T07:18:53.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f948c162580 0x7f948c149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.761 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f948c15e1d0 0x7f948c161000 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 --2- 172.21.15.120:0/951961637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f948c070e10 0x7f948c0c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 >> 172.21.15.120:0/951961637 conn(0x7f948c0d4230 msgr2=0x7f948c0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 shutdown_connections 2024-03-07T07:18:53.762 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.762+0000 7f9492a5e640 1 -- 172.21.15.120:0/951961637 wait complete. 2024-03-07T07:18:53.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new da015db4-af8c-4064-8f8a-c6fce89f0476 6 2024-03-07T07:18:53.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.873+0000 7fb9f075d640 1 Processor -- start 2024-03-07T07:18:53.872 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.873+0000 7fb9f075d640 1 -- start start 2024-03-07T07:18:53.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.874+0000 7fb9f075d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb9e80b3140 0x7fb9e80b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.874+0000 7fb9f075d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e815c580 con 0x7fb9e80b4c10 2024-03-07T07:18:53.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9f075d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e815c860 con 0x7fb9e80b0390 2024-03-07T07:18:53.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9f075d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e815d190 con 0x7fb9e80b3140 2024-03-07T07:18:53.874 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9eecfa640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb9e80b3140 0x7fb9e80b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9eecfa640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb9e80b3140 0x7fb9e80b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39754/0 (socket says 172.21.15.120:39754) 2024-03-07T07:18:53.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9edcf8640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb9e80b0390 0x7fb9e80b2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:60180/0 (socket says 172.21.15.120:60180) 2024-03-07T07:18:53.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9eecfa640 1 -- 172.21.15.120:0/1137955004 learned_addr learned my addr 172.21.15.120:0/1137955004 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9eecfa640 1 -- 172.21.15.120:0/1137955004 >> v1:172.21.15.120:6789/0 conn(0x7fb9e80b0390 legacy=0x7fb9e80b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9eecfa640 1 -- 172.21.15.120:0/1137955004 >> v1:172.21.15.64:6789/0 conn(0x7fb9e80b4c10 legacy=0x7fb9e80b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9ed4f7640 1 -- 172.21.15.120:0/1137955004 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439880277 0 0) 0x7fb9e815c860 con 0x7fb9e80b0390 2024-03-07T07:18:53.875 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.875+0000 7fb9eecfa640 1 -- 172.21.15.120:0/1137955004 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e815cb40 con 0x7fb9e80b3140 2024-03-07T07:18:53.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.876+0000 7fb9eecfa640 1 --2- 172.21.15.120:0/1137955004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b3140 0x7fb9e80b3da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009e50 tx=0x7fb9e002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=899698da55c215b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.876+0000 7fb9ed4f7640 1 -- 172.21.15.120:0/1137955004 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e002cb20 con 0x7fb9e80b3140 2024-03-07T07:18:53.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.876+0000 7fb9ed4f7640 1 -- 172.21.15.120:0/1137955004 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e002ccc0 con 0x7fb9e80b3140 2024-03-07T07:18:53.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.876+0000 7fb9ed4f7640 1 -- 172.21.15.120:0/1137955004 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0034610 con 0x7fb9e80b3140 2024-03-07T07:18:53.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.877+0000 7fb9f075d640 1 -- 172.21.15.120:0/1137955004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b3140 msgr2=0x7fb9e80b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.877+0000 7fb9f075d640 1 --2- 172.21.15.120:0/1137955004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b3140 0x7fb9e80b3da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009e50 tx=0x7fb9e002bf80 comp rx=0 tx=0).stop 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.877+0000 7fb9f075d640 1 -- 172.21.15.120:0/1137955004 shutdown_connections 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.877+0000 7fb9f075d640 1 --2- 172.21.15.120:0/1137955004 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b3140 0x7fb9e80b3da0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.877+0000 7fb9f075d640 1 -- 172.21.15.120:0/1137955004 >> 172.21.15.120:0/1137955004 conn(0x7fb9e806b040 msgr2=0x7fb9e806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.877+0000 7fb9f075d640 1 -- 172.21.15.120:0/1137955004 shutdown_connections 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.877+0000 7fb9f075d640 1 -- 172.21.15.120:0/1137955004 wait complete. 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.879+0000 7fb9f075d640 1 Processor -- start 2024-03-07T07:18:53.877 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.879+0000 7fb9f075d640 1 -- start start 2024-03-07T07:18:53.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9f075d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b0390 0x7fb9e81f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9f075d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9e80b3140 0x7fb9e80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9f075d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb9e80b4c10 0x7fb9e815df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:53.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9f075d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e815fc90 con 0x7fb9e80b3140 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9f075d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e815fb10 con 0x7fb9e80b0390 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9f075d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e815fe10 con 0x7fb9e80b4c10 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9eecfa640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb9e80b4c10 0x7fb9e815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9ee4f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b0390 0x7fb9e81f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9eecfa640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb9e80b4c10 0x7fb9e815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:35800/0 (socket says 172.21.15.120:35800) 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9eecfa640 1 -- 172.21.15.120:0/2521262546 learned_addr learned my addr 172.21.15.120:0/2521262546 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9edcf8640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9e80b3140 0x7fb9e80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9ee4f9640 1 -- 172.21.15.120:0/2521262546 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb9e80b4c10 msgr2=0x7fb9e815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9ee4f9640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb9e80b4c10 0x7fb9e815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9ee4f9640 1 -- 172.21.15.120:0/2521262546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9e80b3140 msgr2=0x7fb9e80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9ee4f9640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9e80b3140 0x7fb9e80ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9ee4f9640 1 -- 172.21.15.120:0/2521262546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e81f2140 con 0x7fb9e80b0390 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.880+0000 7fb9ee4f9640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b0390 0x7fb9e81f1a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800c390 tx=0x7fb9d800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:53.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.881+0000 7fb9deffd640 1 -- 172.21.15.120:0/2521262546 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8017070 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.881+0000 7fb9deffd640 1 -- 172.21.15.120:0/2521262546 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d80052d0 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.881+0000 7fb9deffd640 1 -- 172.21.15.120:0/2521262546 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8003830 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.881+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e81f2420 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.881+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e81f51a0 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.881+0000 7fb9deffd640 1 -- 172.21.15.120:0/2521262546 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb9d801a070 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.881+0000 7fb9deffd640 1 -- 172.21.15.120:0/2521262546 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fb9d8011b30 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.882+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e81f53a0 con 0x7fb9e80b0390 2024-03-07T07:18:53.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.884+0000 7fb9deffd640 1 -- 172.21.15.120:0/2521262546 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb9e81f53a0 con 0x7fb9e80b0390 2024-03-07T07:18:53.956 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.957+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "da015db4-af8c-4064-8f8a-c6fce89f0476", "id": 6} v 0) v1 -- 0x7fb9e81f2940 con 0x7fb9e80b0390 2024-03-07T07:18:53.962 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.963+0000 7fb9deffd640 1 -- 172.21.15.120:0/2521262546 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "da015db4-af8c-4064-8f8a-c6fce89f0476", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb9d8021080 con 0x7fb9e80b0390 2024-03-07T07:18:53.962 INFO:teuthology.orchestra.run.smithi120.stdout:6 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b0390 msgr2=0x7fb9e81f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b0390 0x7fb9e81f1a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800c390 tx=0x7fb9d800c850 comp rx=0 tx=0).stop 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 shutdown_connections 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9e80b3140 0x7fb9e80ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb9e80b4c10 0x7fb9e815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 --2- 172.21.15.120:0/2521262546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9e80b0390 0x7fb9e81f1a10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 >> 172.21.15.120:0/2521262546 conn(0x7fb9e806b040 msgr2=0x7fb9e8074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 shutdown_connections 2024-03-07T07:18:53.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:53.964+0000 7fb9f075d640 1 -- 172.21.15.120:0/2521262546 wait complete. 2024-03-07T07:18:53.973 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 219c980c-69d7-4196-baf2-944263ffa8cf 7 2024-03-07T07:18:53.977 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:53.978+0000 7f8828a3e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T07:18:53.982 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:53.983+0000 7fe7e5c45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T07:18:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.074+0000 7fada3a94640 1 Processor -- start 2024-03-07T07:18:54.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.074+0000 7fada3a94640 1 -- start start 2024-03-07T07:18:54.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada3a94640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c147850 0x7fad9c0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada3a94640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad9c0b2f70 0x7fad9c0ae500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada3a94640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c147c30 con 0x7fad9c0aea40 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada3a94640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c0b33d0 con 0x7fad9c0b2f70 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada3a94640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c15f120 con 0x7fad9c147850 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada1830640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c147850 0x7fad9c0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada102f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad9c0b2f70 0x7fad9c0ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada102f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad9c0b2f70 0x7fad9c0ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39778/0 (socket says 172.21.15.120:39778) 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada102f640 1 -- 172.21.15.120:0/3408986092 learned_addr learned my addr 172.21.15.120:0/3408986092 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada1830640 1 -- 172.21.15.120:0/3408986092 >> v2:172.21.15.120:3300/0 conn(0x7fad9c0b2f70 msgr2=0x7fad9c0ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada1830640 1 --2- 172.21.15.120:0/3408986092 >> v2:172.21.15.120:3300/0 conn(0x7fad9c0b2f70 0x7fad9c0ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada1830640 1 -- 172.21.15.120:0/3408986092 >> v1:172.21.15.64:6789/0 conn(0x7fad9c0aea40 legacy=0x7fad9c06e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada1830640 1 -- 172.21.15.120:0/3408986092 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c15e550 con 0x7fad9c147850 2024-03-07T07:18:54.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.075+0000 7fada102f640 1 --2- 172.21.15.120:0/3408986092 >> v2:172.21.15.120:3300/0 conn(0x7fad9c0b2f70 0x7fad9c0ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.076+0000 7fada1830640 1 --2- 172.21.15.120:0/3408986092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c147850 0x7fad9c0b2a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad980097a0 tx=0x7fad9802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b272f80a6a2bd6c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.076+0000 7fada082e640 1 -- 172.21.15.120:0/3408986092 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98035020 con 0x7fad9c147850 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.076+0000 7fada082e640 1 -- 172.21.15.120:0/3408986092 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad98031830 con 0x7fad9c147850 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.076+0000 7fada082e640 1 -- 172.21.15.120:0/3408986092 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98031c00 con 0x7fad9c147850 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 -- 172.21.15.120:0/3408986092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c147850 msgr2=0x7fad9c0b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 --2- 172.21.15.120:0/3408986092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c147850 0x7fad9c0b2a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad980097a0 tx=0x7fad9802b8d0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 -- 172.21.15.120:0/3408986092 shutdown_connections 2024-03-07T07:18:54.077 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 --2- 172.21.15.120:0/3408986092 >> v2:172.21.15.120:3300/0 conn(0x7fad9c0b2f70 0x7fad9c0ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 --2- 172.21.15.120:0/3408986092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c147850 0x7fad9c0b2a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 -- 172.21.15.120:0/3408986092 >> 172.21.15.120:0/3408986092 conn(0x7fad9c06b040 msgr2=0x7fad9c06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:54.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 -- 172.21.15.120:0/3408986092 shutdown_connections 2024-03-07T07:18:54.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.077+0000 7fada3a94640 1 -- 172.21.15.120:0/3408986092 wait complete. 2024-03-07T07:18:54.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.078+0000 7fada3a94640 1 Processor -- start 2024-03-07T07:18:54.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.078+0000 7fada3a94640 1 -- start start 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada3a94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad9c0aea40 0x7fad9c13f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada3a94640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c0b2f70 0x7fad9c13c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada3a94640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad9c147850 0x7fad9c13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada3a94640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c15edb0 con 0x7fad9c147850 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada3a94640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c15ead0 con 0x7fad9c0aea40 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada3a94640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0aeea0 con 0x7fad9c0b2f70 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad9c0aea40 0x7fad9c13f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada102f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c0b2f70 0x7fad9c13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad9c0aea40 0x7fad9c13f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:39790/0 (socket says 172.21.15.120:39790) 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 -- 172.21.15.120:0/2475201737 learned_addr learned my addr 172.21.15.120:0/2475201737 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada2031640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad9c147850 0x7fad9c13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 -- 172.21.15.120:0/2475201737 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c0b2f70 msgr2=0x7fad9c13c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c0b2f70 0x7fad9c13c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 -- 172.21.15.120:0/2475201737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad9c147850 msgr2=0x7fad9c13cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad9c147850 0x7fad9c13cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 -- 172.21.15.120:0/2475201737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c13fb90 con 0x7fad9c0aea40 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada1830640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad9c0aea40 0x7fad9c13f650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fad98034040 tx=0x7fad980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fad927fc640 1 -- 172.21.15.120:0/2475201737 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98040070 con 0x7fad9c0aea40 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fad927fc640 1 -- 172.21.15.120:0/2475201737 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9803a910 con 0x7fad9c0aea40 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c0b1740 con 0x7fad9c0aea40 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.079+0000 7fad927fc640 1 -- 172.21.15.120:0/2475201737 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98035040 con 0x7fad9c0aea40 2024-03-07T07:18:54.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.080+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c0b1d00 con 0x7fad9c0aea40 2024-03-07T07:18:54.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.080+0000 7fad927fc640 1 -- 172.21.15.120:0/2475201737 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fad98038020 con 0x7fad9c0aea40 2024-03-07T07:18:54.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.080+0000 7fad927fc640 1 -- 172.21.15.120:0/2475201737 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fad98039a90 con 0x7fad9c0aea40 2024-03-07T07:18:54.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.080+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c062c90 con 0x7fad9c0aea40 2024-03-07T07:18:54.082 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.082+0000 7fad927fc640 1 -- 172.21.15.120:0/2475201737 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fad98039d30 con 0x7fad9c0aea40 2024-03-07T07:18:54.118 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.119+0000 7f8828a3e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T07:18:54.124 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.124+0000 7fe7e5c45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T07:18:54.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.153+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "219c980c-69d7-4196-baf2-944263ffa8cf", "id": 7} v 0) v1 -- 0x7fad9c0b4a50 con 0x7fad9c0aea40 2024-03-07T07:18:54.158 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.159+0000 7fad927fc640 1 -- 172.21.15.120:0/2475201737 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "219c980c-69d7-4196-baf2-944263ffa8cf", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fad980449d0 con 0x7fad9c0aea40 2024-03-07T07:18:54.158 INFO:teuthology.orchestra.run.smithi120.stdout:7 2024-03-07T07:18:54.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad9c0aea40 msgr2=0x7fad9c13f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad9c0aea40 0x7fad9c13f650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fad98034040 tx=0x7fad980046c0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 shutdown_connections 2024-03-07T07:18:54.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad9c147850 0x7fad9c13cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad9c0b2f70 0x7fad9c13c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 --2- 172.21.15.120:0/2475201737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad9c0aea40 0x7fad9c13f650 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 >> 172.21.15.120:0/2475201737 conn(0x7fad9c06b040 msgr2=0x7fad9c162270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:54.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 shutdown_connections 2024-03-07T07:18:54.160 INFO:teuthology.orchestra.run.smithi120.stderr:2024-03-07T07:18:54.160+0000 7fada3a94640 1 -- 172.21.15.120:0/2475201737 wait complete. 2024-03-07T07:18:54.169 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-07T07:18:54.169 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-07T07:18:54.171 INFO:tasks.ceph.osd.0:Started 2024-03-07T07:18:54.172 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-07T07:18:54.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-07T07:18:54.174 INFO:tasks.ceph.osd.1:Started 2024-03-07T07:18:54.174 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-07T07:18:54.174 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-07T07:18:54.176 INFO:tasks.ceph.osd.2:Started 2024-03-07T07:18:54.176 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-07T07:18:54.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-07T07:18:54.181 INFO:tasks.ceph.osd.3:Started 2024-03-07T07:18:54.181 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-07T07:18:54.181 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-07T07:18:54.212 INFO:tasks.ceph.osd.4:Started 2024-03-07T07:18:54.212 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-07T07:18:54.212 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-07T07:18:54.214 INFO:tasks.ceph.osd.5:Started 2024-03-07T07:18:54.214 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-07T07:18:54.214 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-07T07:18:54.216 INFO:tasks.ceph.osd.6:Started 2024-03-07T07:18:54.216 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-07T07:18:54.216 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-07T07:18:54.218 INFO:tasks.ceph.osd.7:Started 2024-03-07T07:18:54.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:18:54.250 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.251+0000 7f8828a3e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T07:18:54.264 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.265+0000 7fe7e5c45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T07:18:54.316 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.316+0000 7fe7e5c45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T07:18:54.316 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.317+0000 7f8828a3e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T07:18:54.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.337+0000 7f3406e87640 1 Processor -- start 2024-03-07T07:18:54.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.337+0000 7f3406e87640 1 -- start start 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.337+0000 7f3406e87640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f340013dd70 0x7f340015f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.337+0000 7f3406e87640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340013e150 con 0x7f340015ea40 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.337+0000 7f3406e87640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340006e3f0 con 0x7f34000b31e0 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.337+0000 7f3406e87640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340006f000 con 0x7f340013dd70 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f33fffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f340015ea40 0x7f34000b2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44570/0 (socket says 172.21.15.64:44570) 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f33fffff640 1 -- 172.21.15.64:0/2457804790 learned_addr learned my addr 172.21.15.64:0/2457804790 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f3404c23640 1 --2- 172.21.15.64:0/2457804790 >> v2:172.21.15.120:3300/0 conn(0x7f340013dd70 0x7f340015f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f33ff7fe640 1 -- 172.21.15.64:0/2457804790 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671339813 0 0) 0x7f340013e150 con 0x7f340015ea40 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f33ff7fe640 1 -- 172.21.15.64:0/2457804790 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f0003520 con 0x7f340015ea40 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f3404c23640 1 -- 172.21.15.64:0/2457804790 >> v1:172.21.15.120:6789/0 conn(0x7f34000b31e0 legacy=0x7f34000cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f3404c23640 1 -- 172.21.15.64:0/2457804790 >> v1:172.21.15.64:6789/0 conn(0x7f340015ea40 legacy=0x7f34000b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.338+0000 7f3404c23640 1 -- 172.21.15.64:0/2457804790 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340006e6d0 con 0x7f340013dd70 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.339+0000 7f3404c23640 1 --2- 172.21.15.64:0/2457804790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f340013dd70 0x7f340015f270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33f40097a0 tx=0x7f33f402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd091aa74d483a03 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.339+0000 7f33ff7fe640 1 -- 172.21.15.64:0/2457804790 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4034070 con 0x7f340013dd70 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.339+0000 7f33ff7fe640 1 -- 172.21.15.64:0/2457804790 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4030610 con 0x7f340013dd70 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.339+0000 7f33ff7fe640 1 -- 172.21.15.64:0/2457804790 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40309e0 con 0x7f340013dd70 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.339+0000 7f3406e87640 1 -- 172.21.15.64:0/2457804790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f340013dd70 msgr2=0x7f340015f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.340+0000 7f3406e87640 1 --2- 172.21.15.64:0/2457804790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f340013dd70 0x7f340015f270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33f40097a0 tx=0x7f33f402b960 comp rx=0 tx=0).stop 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.340+0000 7f3406e87640 1 -- 172.21.15.64:0/2457804790 shutdown_connections 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.340+0000 7f3406e87640 1 --2- 172.21.15.64:0/2457804790 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f340013dd70 0x7f340015f270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.340+0000 7f3406e87640 1 -- 172.21.15.64:0/2457804790 >> 172.21.15.64:0/2457804790 conn(0x7f340006b040 msgr2=0x7f340006b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:54.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.340+0000 7f3406e87640 1 -- 172.21.15.64:0/2457804790 shutdown_connections 2024-03-07T07:18:54.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.340+0000 7f3406e87640 1 -- 172.21.15.64:0/2457804790 wait complete. 2024-03-07T07:18:54.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.341+0000 7f3406e87640 1 Processor -- start 2024-03-07T07:18:54.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.341+0000 7f3406e87640 1 -- start start 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3406e87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34000b31e0 0x7f3400159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3406e87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340013dd70 0x7f3400145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3406e87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f340015ea40 0x7f3400155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3406e87640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340015f7b0 con 0x7f340013dd70 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3406e87640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340006ec90 con 0x7f34000b31e0 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3406e87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000cfef0 con 0x7f340015ea40 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340013dd70 0x7f3400145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340013dd70 0x7f3400145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49680/0 (socket says 172.21.15.64:49680) 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 -- 172.21.15.64:0/4238338190 learned_addr learned my addr 172.21.15.64:0/4238338190 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3404c23640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34000b31e0 0x7f3400159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3405424640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f340015ea40 0x7f3400155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 -- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f340015ea40 msgr2=0x7f3400155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f340015ea40 0x7f3400155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 -- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34000b31e0 msgr2=0x7f3400159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34000b31e0 0x7f3400159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 -- 172.21.15.64:0/4238338190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340015a380 con 0x7f340013dd70 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3404c23640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34000b31e0 0x7f3400159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fffff640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340013dd70 0x7f3400145aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c410 tx=0x7f33ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fd7fa640 1 -- 172.21.15.64:0/4238338190 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec017070 con 0x7f340013dd70 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fd7fa640 1 -- 172.21.15.64:0/4238338190 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec005ce0 con 0x7f340013dd70 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340015a660 con 0x7f340013dd70 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.342+0000 7f33fd7fa640 1 -- 172.21.15.64:0/4238338190 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0072f0 con 0x7f340013dd70 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.343+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34001f23a0 con 0x7f340013dd70 2024-03-07T07:18:54.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.343+0000 7f33fd7fa640 1 -- 172.21.15.64:0/4238338190 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f33ec01a070 con 0x7f340013dd70 2024-03-07T07:18:54.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.343+0000 7f33fd7fa640 1 -- 172.21.15.64:0/4238338190 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f33ec012ad0 con 0x7f340013dd70 2024-03-07T07:18:54.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.343+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340015a830 con 0x7f340013dd70 2024-03-07T07:18:54.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.345+0000 7f33fd7fa640 1 -- 172.21.15.64:0/4238338190 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f33ec021020 con 0x7f340013dd70 2024-03-07T07:18:54.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.407+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3400062c90 con 0x7f340013dd70 2024-03-07T07:18:54.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.407+0000 7f33fd7fa640 1 -- 172.21.15.64:0/4238338190 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f33ec007600 con 0x7f340013dd70 2024-03-07T07:18:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:18:54.408 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":10,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:18:54.154592+0000","last_up_change":"0.000000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340013dd70 msgr2=0x7f3400145aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340013dd70 0x7f3400145aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c410 tx=0x7f33ec00c8d0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 shutdown_connections 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f340013dd70 0x7f3400145aa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f340015ea40 0x7f3400155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 --2- 172.21.15.64:0/4238338190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34000b31e0 0x7f3400159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 >> 172.21.15.64:0/4238338190 conn(0x7f340006b040 msgr2=0x7f340013b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 shutdown_connections 2024-03-07T07:18:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:18:54.408+0000 7f3406e87640 1 -- 172.21.15.64:0/4238338190 wait complete. 2024-03-07T07:18:54.419 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-07T07:18:54.419 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-07T07:18:54.428 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.429+0000 7f8828a3e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:18:54.436 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.436+0000 7fe7e5c45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:18:54.523 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.524+0000 7f8828a3e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T07:18:54.535 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.536+0000 7fe7e5c45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T07:18:54.676 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.677+0000 7f8828a3e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:18:54.697 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.697+0000 7fe7e5c45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T07:18:54.719 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.720+0000 7f8828a3e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T07:18:54.741 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.741+0000 7fe7e5c45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T07:18:54.777 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-03-07T07:18:54.777+0000 7f6aece1f640 -1 Falling back to public interface 2024-03-07T07:18:54.790 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-03-07T07:18:54.790+0000 7f8e8880d640 -1 Falling back to public interface 2024-03-07T07:18:54.796 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-03-07T07:18:54.796+0000 7fcce458a640 -1 Falling back to public interface 2024-03-07T07:18:54.801 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-03-07T07:18:54.802+0000 7f4fb9ed9640 -1 Falling back to public interface 2024-03-07T07:18:54.812 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-03-07T07:18:54.813+0000 7efd25e17640 -1 Falling back to public interface 2024-03-07T07:18:54.817 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-03-07T07:18:54.818+0000 7fa1acca0640 -1 Falling back to public interface 2024-03-07T07:18:54.826 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-03-07T07:18:54.827+0000 7f378760e640 -1 Falling back to public interface 2024-03-07T07:18:54.839 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-03-07T07:18:54.841+0000 7f964901a640 -1 Falling back to public interface 2024-03-07T07:18:54.879 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:54.880+0000 7f8828a3e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T07:18:54.908 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:54.908+0000 7fe7e5c45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T07:18:55.223 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-07T07:18:55.314 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.314+0000 7f4ec3e34640 1 Processor -- start 2024-03-07T07:18:55.314 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.314+0000 7f4ec3e34640 1 -- start start 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec3e34640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4ebc072600 0x7f4ebc1416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec3e34640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ebc142130 con 0x7f4ebc06b050 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec3e34640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ebc142410 con 0x7f4ebc141bf0 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec3e34640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ebc1569c0 con 0x7f4ebc072600 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec13cf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4ebc072600 0x7f4ebc1416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec1bd0640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4ebc06b050 0x7f4ebc06b430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44582/0 (socket says 172.21.15.64:44582) 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec1bd0640 1 -- 172.21.15.64:0/3831949113 learned_addr learned my addr 172.21.15.64:0/3831949113 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec13cf640 1 -- 172.21.15.64:0/3831949113 >> v1:172.21.15.120:6789/0 conn(0x7f4ebc141bf0 legacy=0x7f4ebc0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec13cf640 1 -- 172.21.15.64:0/3831949113 >> v1:172.21.15.64:6789/0 conn(0x7f4ebc06b050 legacy=0x7f4ebc06b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:55.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.315+0000 7f4ec13cf640 1 -- 172.21.15.64:0/3831949113 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc071e00 con 0x7f4ebc072600 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.316+0000 7f4ec13cf640 1 --2- 172.21.15.64:0/3831949113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc072600 0x7f4ebc1416b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0097a0 tx=0x7f4eac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e28d46e1caa139 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.316+0000 7f4ec0bce640 1 -- 172.21.15.64:0/3831949113 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac034070 con 0x7f4ebc072600 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.316+0000 7f4ec0bce640 1 -- 172.21.15.64:0/3831949113 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac030610 con 0x7f4ebc072600 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.316+0000 7f4ec0bce640 1 -- 172.21.15.64:0/3831949113 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0309e0 con 0x7f4ebc072600 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.317+0000 7f4ec3e34640 1 -- 172.21.15.64:0/3831949113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc072600 msgr2=0x7f4ebc1416b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.317+0000 7f4ec3e34640 1 --2- 172.21.15.64:0/3831949113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc072600 0x7f4ebc1416b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0097a0 tx=0x7f4eac02b960 comp rx=0 tx=0).stop 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.317+0000 7f4ec3e34640 1 -- 172.21.15.64:0/3831949113 shutdown_connections 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.317+0000 7f4ec3e34640 1 --2- 172.21.15.64:0/3831949113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc072600 0x7f4ebc1416b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.317+0000 7f4ec3e34640 1 -- 172.21.15.64:0/3831949113 >> 172.21.15.64:0/3831949113 conn(0x7f4ebc0b2930 msgr2=0x7f4ebc0b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.317+0000 7f4ec3e34640 1 -- 172.21.15.64:0/3831949113 shutdown_connections 2024-03-07T07:18:55.317 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.317+0000 7f4ec3e34640 1 -- 172.21.15.64:0/3831949113 wait complete. 2024-03-07T07:18:55.318 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.318+0000 7f4ec3e34640 1 Processor -- start 2024-03-07T07:18:55.318 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.318+0000 7f4ec3e34640 1 -- start start 2024-03-07T07:18:55.320 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec3e34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc06b050 0x7f4ebc0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:55.320 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec3e34640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ebc072600 0x7f4ebc15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:55.320 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec3e34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ebc141bf0 0x7f4ebc15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:18:55.320 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec3e34640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc13f200 con 0x7f4ebc06b050 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec3e34640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc15c740 con 0x7f4ebc141bf0 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec3e34640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc13f380 con 0x7f4ebc072600 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc06b050 0x7f4ebc0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc06b050 0x7f4ebc0b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49696/0 (socket says 172.21.15.64:49696) 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 -- 172.21.15.64:0/1304131529 learned_addr learned my addr 172.21.15.64:0/1304131529 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec23d1640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ebc141bf0 0x7f4ebc15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec13cf640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ebc072600 0x7f4ebc15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 -- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ebc072600 msgr2=0x7f4ebc15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ebc072600 0x7f4ebc15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 -- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ebc141bf0 msgr2=0x7f4ebc15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ebc141bf0 0x7f4ebc15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:55.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 -- 172.21.15.64:0/1304131529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0d1320 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec23d1640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ebc141bf0 0x7f4ebc15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec1bd0640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc06b050 0x7f4ebc0b5950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c400 tx=0x7f4eb800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4eb27fc640 1 -- 172.21.15.64:0/1304131529 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8017070 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4eb27fc640 1 -- 172.21.15.64:0/1304131529 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8005ce0 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.319+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc0d0280 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.320+0000 7f4eb27fc640 1 -- 172.21.15.64:0/1304131529 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80072f0 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.320+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc0d0840 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.320+0000 7f4eb27fc640 1 -- 172.21.15.64:0/1304131529 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4eb801a070 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.320+0000 7f4eb27fc640 1 -- 172.21.15.64:0/1304131529 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4eb8012ad0 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.320+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc068570 con 0x7f4ebc06b050 2024-03-07T07:18:55.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.323+0000 7f4eb27fc640 1 -- 172.21.15.64:0/1304131529 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4eb8021020 con 0x7f4ebc06b050 2024-03-07T07:18:55.385 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.385+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ebc04f3c0 con 0x7f4ebc06b050 2024-03-07T07:18:55.385 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.386+0000 7f4eb27fc640 1 -- 172.21.15.64:0/1304131529 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f4eb8007600 con 0x7f4ebc06b050 2024-03-07T07:18:55.386 INFO:teuthology.misc.health.smithi064.stdout: 2024-03-07T07:18:55.386 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":10,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:18:54.154592+0000","last_up_change":"0.000000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.386+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc06b050 msgr2=0x7f4ebc0b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.386+0000 7f4ec3e34640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc06b050 0x7f4ebc0b5950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c400 tx=0x7f4eb800c8c0 comp rx=0 tx=0).stop 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.387+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 shutdown_connections 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.387+0000 7f4ec3e34640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ebc141bf0 0x7f4ebc15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.387+0000 7f4ec3e34640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ebc072600 0x7f4ebc15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.387+0000 7f4ec3e34640 1 --2- 172.21.15.64:0/1304131529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ebc06b050 0x7f4ebc0b5950 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.387+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 >> 172.21.15.64:0/1304131529 conn(0x7f4ebc0b2930 msgr2=0x7f4ebc13f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.387+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 shutdown_connections 2024-03-07T07:18:55.387 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:18:55.387+0000 7f4ec3e34640 1 -- 172.21.15.64:0/1304131529 wait complete. 2024-03-07T07:18:55.396 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-07T07:18:55.712 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:55.712+0000 7f8828a3e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T07:18:55.757 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:55.758+0000 7fe7e5c45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T07:18:55.988 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:55.988+0000 7f8828a3e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T07:18:56.039 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:56.039+0000 7f8828a3e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T07:18:56.046 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:56.046+0000 7fe7e5c45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T07:18:56.099 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:56.100+0000 7fe7e5c45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T07:18:56.152 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:56.152+0000 7f8828a3e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T07:18:56.220 INFO:tasks.ceph.mgr.y.smithi064.stderr:2024-03-07T07:18:56.220+0000 7fe7e5c45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T07:18:57.181 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:57.181+0000 7f87c9508640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-07T07:18:57.181 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-03-07T07:18:57.182+0000 7f87c9508640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-07T07:18:58.263 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-03-07T07:18:58.263+0000 7f6aece1f640 -1 osd.0 0 log_to_monitors true 2024-03-07T07:18:58.302 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-03-07T07:18:58.302+0000 7f4fb9ed9640 -1 osd.3 0 log_to_monitors true 2024-03-07T07:18:58.310 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-03-07T07:18:58.310+0000 7efd25e17640 -1 osd.5 0 log_to_monitors true 2024-03-07T07:18:58.320 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-03-07T07:18:58.320+0000 7f8e8880d640 -1 osd.1 0 log_to_monitors true 2024-03-07T07:18:58.332 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-03-07T07:18:58.332+0000 7fcce458a640 -1 osd.2 0 log_to_monitors true 2024-03-07T07:18:58.342 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-03-07T07:18:58.343+0000 7f378760e640 -1 osd.7 0 log_to_monitors true 2024-03-07T07:18:58.358 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-03-07T07:18:58.358+0000 7f964901a640 -1 osd.6 0 log_to_monitors true 2024-03-07T07:18:58.361 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-03-07T07:18:58.361+0000 7fa1acca0640 -1 osd.4 0 log_to_monitors true 2024-03-07T07:19:02.202 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:02.218 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-03-07T07:19:02.218+0000 7f4fb53f6640 -1 osd.3 0 waiting for initial osdmap 2024-03-07T07:19:02.232 INFO:tasks.ceph.osd.3.smithi064.stderr:2024-03-07T07:19:02.232+0000 7f4fada80640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.242 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-03-07T07:19:02.243+0000 7f6ae7dd4640 -1 osd.0 0 waiting for initial osdmap 2024-03-07T07:19:02.254 INFO:tasks.ceph.osd.0.smithi064.stderr:2024-03-07T07:19:02.255+0000 7f6ae0406640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.283 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-03-07T07:19:02.283+0000 7fccdf9f6640 -1 osd.2 0 waiting for initial osdmap 2024-03-07T07:19:02.295 INFO:tasks.ceph.osd.2.smithi064.stderr:2024-03-07T07:19:02.296+0000 7fccd8080640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.300 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-03-07T07:19:02.300+0000 7fa1a99f9640 -1 osd.4 0 waiting for initial osdmap 2024-03-07T07:19:02.302 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-03-07T07:19:02.303+0000 7f9643df6640 -1 osd.6 0 waiting for initial osdmap 2024-03-07T07:19:02.306 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-03-07T07:19:02.306+0000 7f37825aa640 -1 osd.7 0 waiting for initial osdmap 2024-03-07T07:19:02.312 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.313+0000 7f1c0c7a7640 1 Processor -- start 2024-03-07T07:19:02.313 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.313+0000 7f1c0c7a7640 1 -- start start 2024-03-07T07:19:02.314 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0c7a7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1c04161650 0x7f1c040d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:02.314 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0c7a7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c040d2470 con 0x7f1c04161a30 2024-03-07T07:19:02.314 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0c7a7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c040d2750 con 0x7f1c040d0f90 2024-03-07T07:19:02.314 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0c7a7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c040d2a30 con 0x7f1c04161650 2024-03-07T07:19:02.314 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c09d42640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1c04161a30 0x7f1c040d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44842/0 (socket says 172.21.15.64:44842) 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c09d42640 1 -- 172.21.15.64:0/4130855089 learned_addr learned my addr 172.21.15.64:0/4130855089 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0a543640 1 --2- 172.21.15.64:0/4130855089 >> v2:172.21.15.120:3300/0 conn(0x7f1c04161650 0x7f1c040d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c09541640 1 -- 172.21.15.64:0/4130855089 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221980214 0 0) 0x7f1c040d2470 con 0x7f1c04161a30 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c09541640 1 -- 172.21.15.64:0/4130855089 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf0003520 con 0x7f1c04161a30 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0a543640 1 -- 172.21.15.64:0/4130855089 >> v1:172.21.15.120:6789/0 conn(0x7f1c040d0f90 legacy=0x7f1c040b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0a543640 1 -- 172.21.15.64:0/4130855089 >> v1:172.21.15.64:6789/0 conn(0x7f1c04161a30 legacy=0x7f1c040d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.314+0000 7f1c0a543640 1 -- 172.21.15.64:0/4130855089 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040d2d10 con 0x7f1c04161650 2024-03-07T07:19:02.315 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.315+0000 7f1c0a543640 1 --2- 172.21.15.64:0/4130855089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 0x7f1c040d49b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c000097a0 tx=0x7f1c0002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6898226615f0e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:02.316 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-03-07T07:19:02.316+0000 7fa1a0828640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.319 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-03-07T07:19:02.320+0000 7f963c480640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.320+0000 7f1c09541640 1 -- 172.21.15.64:0/4130855089 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00034070 con 0x7f1c04161650 2024-03-07T07:19:02.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.320+0000 7f1c09541640 1 -- 172.21.15.64:0/4130855089 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00030610 con 0x7f1c04161650 2024-03-07T07:19:02.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.320+0000 7f1c09541640 1 -- 172.21.15.64:0/4130855089 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c000309e0 con 0x7f1c04161650 2024-03-07T07:19:02.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.321+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/4130855089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 msgr2=0x7f1c040d49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:02.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.321+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/4130855089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 0x7f1c040d49b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c000097a0 tx=0x7f1c0002b960 comp rx=0 tx=0).stop 2024-03-07T07:19:02.321 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.321+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/4130855089 shutdown_connections 2024-03-07T07:19:02.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.321+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/4130855089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 0x7f1c040d49b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.321+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/4130855089 >> 172.21.15.64:0/4130855089 conn(0x7f1c040d3e90 msgr2=0x7f1c040d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:02.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.321+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/4130855089 shutdown_connections 2024-03-07T07:19:02.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.321+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/4130855089 wait complete. 2024-03-07T07:19:02.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.322+0000 7f1c0c7a7640 1 Processor -- start 2024-03-07T07:19:02.322 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.322+0000 7f1c0c7a7640 1 -- start start 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.323+0000 7f1c0c7a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c040d0f90 0x7f1c041544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.323+0000 7f1c0c7a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 0x7f1c040c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.323+0000 7f1c0c7a7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c04161a30 0x7f1c0415fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0c7a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040b4ee0 con 0x7f1c040d0f90 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0c7a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040b4d60 con 0x7f1c04161650 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0c7a7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040b5060 con 0x7f1c04161a30 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c040d0f90 0x7f1c041544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:02.325 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c040d0f90 0x7f1c041544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59618/0 (socket says 172.21.15.64:59618) 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 -- 172.21.15.64:0/3097565571 learned_addr learned my addr 172.21.15.64:0/3097565571 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c09d42640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 0x7f1c040c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0ad44640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c04161a30 0x7f1c0415fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 -- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c04161a30 msgr2=0x7f1c0415fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c04161a30 0x7f1c0415fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 -- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 msgr2=0x7f1c040c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 0x7f1c040c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 -- 172.21.15.64:0/3097565571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040c7c80 con 0x7f1c040d0f90 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0a543640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c040d0f90 0x7f1c041544a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098c0 tx=0x7f1c000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1bfaffd640 1 -- 172.21.15.64:0/3097565571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00034040 con 0x7f1c040d0f90 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1bfaffd640 1 -- 172.21.15.64:0/3097565571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0003aaa0 con 0x7f1c040d0f90 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1bfaffd640 1 -- 172.21.15.64:0/3097565571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c000443f0 con 0x7f1c040d0f90 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c040c7f60 con 0x7f1c040d0f90 2024-03-07T07:19:02.326 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.324+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c040c8520 con 0x7f1c040d0f90 2024-03-07T07:19:02.327 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.325+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c04068570 con 0x7f1c040d0f90 2024-03-07T07:19:02.327 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.326+0000 7f1bfaffd640 1 -- 172.21.15.64:0/3097565571 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 89722+0+0 (secure 0 0 0) 0x7f1c0004b060 con 0x7f1c040d0f90 2024-03-07T07:19:02.327 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.327+0000 7f1bfaffd640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1be806b440 0x7f1be806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:02.327 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.327+0000 7f1bfaffd640 1 -- 172.21.15.64:0/3097565571 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f1c000aaf50 con 0x7f1c040d0f90 2024-03-07T07:19:02.328 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.327+0000 7f1c09d42640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1be806b440 0x7f1be806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:02.328 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.327+0000 7f1c09d42640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1be806b440 0x7f1be806d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1c040b10d0 tx=0x7f1bf4007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:02.328 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-03-07T07:19:02.328+0000 7f377abdc640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.331 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.331+0000 7f1bfaffd640 1 -- 172.21.15.64:0/3097565571 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1c0003f070 con 0x7f1c040d0f90 2024-03-07T07:19:02.332 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-03-07T07:19:02.332+0000 7efd20df6640 -1 osd.5 0 waiting for initial osdmap 2024-03-07T07:19:02.344 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-03-07T07:19:02.345+0000 7efd19480640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.349 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-03-07T07:19:02.349+0000 7f8e835f6640 -1 osd.1 0 waiting for initial osdmap 2024-03-07T07:19:02.361 INFO:tasks.ceph.osd.1.smithi064.stderr:2024-03-07T07:19:02.362+0000 7f8e7bc80640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T07:19:02.521 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.521+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c040b99a0 con 0x7f1c040d0f90 2024-03-07T07:19:02.522 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.522+0000 7f1bfaffd640 1 -- 172.21.15.64:0/3097565571 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f1c0007a680 con 0x7f1c040d0f90 2024-03-07T07:19:02.522 INFO:teuthology.misc.health.smithi064.stdout: 2024-03-07T07:19:02.522 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":12,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:00.182828+0000","last_up_change":"0.000000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:02.524 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1be806b440 msgr2=0x7f1be806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1be806b440 0x7f1be806d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1c040b10d0 tx=0x7f1bf4007d30 comp rx=0 tx=0).stop 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c040d0f90 msgr2=0x7f1c041544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c040d0f90 0x7f1c041544a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098c0 tx=0x7f1c000046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 shutdown_connections 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1be806b440 0x7f1be806d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c04161a30 0x7f1c0415fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c04161650 0x7f1c040c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 --2- 172.21.15.64:0/3097565571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c040d0f90 0x7f1c041544a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 >> 172.21.15.64:0/3097565571 conn(0x7f1c040d3e90 msgr2=0x7f1c0415f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 shutdown_connections 2024-03-07T07:19:02.525 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:02.524+0000 7f1c0c7a7640 1 -- 172.21.15.64:0/3097565571 wait complete. 2024-03-07T07:19:02.534 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-07T07:19:09.341 DEBUG:teuthology.orchestra.run.smithi064:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:09.442 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.442+0000 7fe527310640 1 Processor -- start 2024-03-07T07:19:09.442 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.442+0000 7fe527310640 1 -- start start 2024-03-07T07:19:09.443 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe527310640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe5200c8340 0x7fe5200d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.443 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe527310640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.443 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe527310640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200d1210 con 0x7fe52006c500 2024-03-07T07:19:09.443 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe527310640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5200cca40 con 0x7fe5200c8340 2024-03-07T07:19:09.443 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe527310640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200d14f0 con 0x7fe52006d260 2024-03-07T07:19:09.443 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5250ac640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe5200c8340 0x7fe5200d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.443 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5250ac640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fe5200c8340 0x7fe5200d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59632/0 (socket says 172.21.15.64:59632) 2024-03-07T07:19:09.444 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5250ac640 1 -- 172.21.15.64:0/2779040018 learned_addr learned my addr 172.21.15.64:0/2779040018 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:09.444 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5258ad640 1 --2- 172.21.15.64:0/2779040018 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.444 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5250ac640 1 -- 172.21.15.64:0/2779040018 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 msgr2=0x7fe5200cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:09.444 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5250ac640 1 --2- 172.21.15.64:0/2779040018 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.444 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5250ac640 1 -- 172.21.15.64:0/2779040018 >> v1:172.21.15.120:6789/0 conn(0x7fe52006c500 legacy=0x7fe52006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:09.444 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.443+0000 7fe5250ac640 1 -- 172.21.15.64:0/2779040018 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200cc3f0 con 0x7fe5200c8340 2024-03-07T07:19:09.445 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.444+0000 7fe5250ac640 1 --2- 172.21.15.64:0/2779040018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5200c8340 0x7fe5200d0990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009870 tx=0x7fe51c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fed50634abde251 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:09.445 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.444+0000 7fe517fff640 1 -- 172.21.15.64:0/2779040018 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c035020 con 0x7fe5200c8340 2024-03-07T07:19:09.445 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.444+0000 7fe517fff640 1 -- 172.21.15.64:0/2779040018 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c031640 con 0x7fe5200c8340 2024-03-07T07:19:09.445 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.444+0000 7fe517fff640 1 -- 172.21.15.64:0/2779040018 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c02d040 con 0x7fe5200c8340 2024-03-07T07:19:09.445 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 -- 172.21.15.64:0/2779040018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5200c8340 msgr2=0x7fe5200d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.446 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 --2- 172.21.15.64:0/2779040018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5200c8340 0x7fe5200d0990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009870 tx=0x7fe51c02b8f0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.446 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 -- 172.21.15.64:0/2779040018 shutdown_connections 2024-03-07T07:19:09.446 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 --2- 172.21.15.64:0/2779040018 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.446 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 --2- 172.21.15.64:0/2779040018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5200c8340 0x7fe5200d0990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.446 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 -- 172.21.15.64:0/2779040018 >> 172.21.15.64:0/2779040018 conn(0x7fe5200d1c70 msgr2=0x7fe5200d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:09.446 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 -- 172.21.15.64:0/2779040018 shutdown_connections 2024-03-07T07:19:09.446 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.445+0000 7fe527310640 1 -- 172.21.15.64:0/2779040018 wait complete. 2024-03-07T07:19:09.448 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.446+0000 7fe527310640 1 Processor -- start 2024-03-07T07:19:09.448 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.446+0000 7fe527310640 1 -- start start 2024-03-07T07:19:09.448 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe527310640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe52006c500 0x7fe5201de8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.448 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe527310640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.448 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe527310640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5200c8340 0x7fe5200c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.448 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe527310640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200cd410 con 0x7fe52006c500 2024-03-07T07:19:09.448 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe527310640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200d0ed0 con 0x7fe5200c8340 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe527310640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200cd590 con 0x7fe52006d260 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe52006c500 0x7fe5201de8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe52006c500 0x7fe5201de8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59638/0 (socket says 172.21.15.64:59638) 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 -- 172.21.15.64:0/4089976419 learned_addr learned my addr 172.21.15.64:0/4089976419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5258ad640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5200c8340 0x7fe5200c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5248ab640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 -- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 msgr2=0x7fe5200cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 -- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5200c8340 msgr2=0x7fe5200c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5200c8340 0x7fe5200c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.447+0000 7fe5250ac640 1 -- 172.21.15.64:0/4089976419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5201df340 con 0x7fe52006c500 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.448+0000 7fe5250ac640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe52006c500 0x7fe5201de8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000c00 tx=0x7fe51c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.448+0000 7fe515ffb640 1 -- 172.21.15.64:0/4089976419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c040070 con 0x7fe52006c500 2024-03-07T07:19:09.449 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.448+0000 7fe515ffb640 1 -- 172.21.15.64:0/4089976419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c0399f0 con 0x7fe52006c500 2024-03-07T07:19:09.450 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.448+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5201dee00 con 0x7fe52006c500 2024-03-07T07:19:09.450 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.448+0000 7fe515ffb640 1 -- 172.21.15.64:0/4089976419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c035040 con 0x7fe52006c500 2024-03-07T07:19:09.450 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.448+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5201e2c70 con 0x7fe52006c500 2024-03-07T07:19:09.450 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.449+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe520068570 con 0x7fe52006c500 2024-03-07T07:19:09.451 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.449+0000 7fe515ffb640 1 -- 172.21.15.64:0/4089976419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe51c04b060 con 0x7fe52006c500 2024-03-07T07:19:09.451 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.450+0000 7fe515ffb640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe50406b4d0 0x7fe50406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.451 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.450+0000 7fe515ffb640 1 -- 172.21.15.64:0/4089976419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe51c0ab730 con 0x7fe52006c500 2024-03-07T07:19:09.451 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.450+0000 7fe5248ab640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe50406b4d0 0x7fe50406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.451 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.451+0000 7fe5248ab640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe50406b4d0 0x7fe50406d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe5200caf50 tx=0x7fe510009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:09.454 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.454+0000 7fe515ffb640 1 -- 172.21.15.64:0/4089976419 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe5201df340 con 0x7fe52006c500 2024-03-07T07:19:09.636 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.636+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe52006b420 con 0x7fe52006c500 2024-03-07T07:19:09.636 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.637+0000 7fe515ffb640 1 -- 172.21.15.64:0/4089976419 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7fe52006b420 con 0x7fe52006c500 2024-03-07T07:19:09.637 INFO:teuthology.misc.health.smithi064.stdout: 2024-03-07T07:19:09.637 INFO:teuthology.misc.health.smithi064.stdout:{"epoch":16,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:06.250188+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe50406b4d0 msgr2=0x7fe50406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe50406b4d0 0x7fe50406d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe5200caf50 tx=0x7fe510009040 comp rx=0 tx=0).stop 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe52006c500 msgr2=0x7fe5201de8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe52006c500 0x7fe5201de8c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000c00 tx=0x7fe51c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 shutdown_connections 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe50406b4d0 0x7fe50406d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5200c8340 0x7fe5200c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe52006d260 0x7fe5200cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.639 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 --2- 172.21.15.64:0/4089976419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe52006c500 0x7fe5201de8c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.640 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 >> 172.21.15.64:0/4089976419 conn(0x7fe5200d1c70 msgr2=0x7fe5201dfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:09.640 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 shutdown_connections 2024-03-07T07:19:09.640 INFO:teuthology.misc.health.smithi064.stderr:2024-03-07T07:19:09.639+0000 7fe527310640 1 -- 172.21.15.64:0/4089976419 wait complete. 2024-03-07T07:19:09.648 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-07T07:19:09.648 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-07T07:19:09.649 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-07T07:19:09.649 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-07T07:19:09.651 INFO:tasks.ceph.mds.a:Started 2024-03-07T07:19:09.651 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-07T07:19:09.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-07T07:19:09.653 INFO:tasks.ceph.mds.c:Started 2024-03-07T07:19:09.653 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-07T07:19:09.653 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-07T07:19:09.656 INFO:tasks.ceph.mds.b:Started 2024-03-07T07:19:09.656 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-07T07:19:09.656 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-07T07:19:09.658 INFO:tasks.ceph.mds.d:Started 2024-03-07T07:19:09.658 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-07T07:19:09.661 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:09.727 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2024-03-07T07:19:09.728 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2024-03-07T07:19:09.741 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-03-07T07:19:09.741 INFO:tasks.ceph.mds.d.smithi120.stdout:starting mds.d at 2024-03-07T07:19:09.742 INFO:tasks.ceph.mds.b.smithi120.stderr:2024-03-07T07:19:09.742+0000 7fd3e99fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-07T07:19:09.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbefb0dd640 1 Processor -- start 2024-03-07T07:19:09.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbefb0dd640 1 -- start start 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbefb0dd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbef4145350 0x7fbef40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbefb0dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbef4070b50 0x7fbef406eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbefb0dd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef415ffc0 con 0x7fbef415c580 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbefb0dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef40cf1f0 con 0x7fbef4070b50 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbefb0dd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef41602a0 con 0x7fbef4145350 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbef967a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbef4070b50 0x7fbef406eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.767+0000 7fbef3fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbef415c580 0x7fbef4070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44852/0 (socket says 172.21.15.64:44852) 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef3fff640 1 -- 172.21.15.64:0/1155682150 learned_addr learned my addr 172.21.15.64:0/1155682150 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef8e79640 1 --2- 172.21.15.64:0/1155682150 >> v2:172.21.15.120:3300/0 conn(0x7fbef4145350 0x7fbef40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef967a640 1 -- 172.21.15.64:0/1155682150 >> v2:172.21.15.120:3300/0 conn(0x7fbef4145350 msgr2=0x7fbef40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef967a640 1 --2- 172.21.15.64:0/1155682150 >> v2:172.21.15.120:3300/0 conn(0x7fbef4145350 0x7fbef40cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef967a640 1 -- 172.21.15.64:0/1155682150 >> v1:172.21.15.64:6789/0 conn(0x7fbef415c580 legacy=0x7fbef4070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef967a640 1 -- 172.21.15.64:0/1155682150 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4160580 con 0x7fbef4070b50 2024-03-07T07:19:09.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef37fe640 1 -- 172.21.15.64:0/1155682150 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718963612 0 0) 0x7fbef415ffc0 con 0x7fbef415c580 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.768+0000 7fbef967a640 1 --2- 172.21.15.64:0/1155682150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef406eb20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbef40b3b50 tx=0x7fbeec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a394905cced3c538 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbef37fe640 1 -- 172.21.15.64:0/1155682150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec039070 con 0x7fbef4070b50 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbef37fe640 1 -- 172.21.15.64:0/1155682150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec02cc80 con 0x7fbef4070b50 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbef37fe640 1 -- 172.21.15.64:0/1155682150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec034660 con 0x7fbef4070b50 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1155682150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 msgr2=0x7fbef406eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1155682150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef406eb20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbef40b3b50 tx=0x7fbeec02bed0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1155682150 shutdown_connections 2024-03-07T07:19:09.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1155682150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef406eb20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1155682150 >> v2:172.21.15.120:3300/0 conn(0x7fbef4145350 0x7fbef40cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1155682150 >> 172.21.15.64:0/1155682150 conn(0x7fbef406b040 msgr2=0x7fbef406b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:09.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1155682150 shutdown_connections 2024-03-07T07:19:09.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.769+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1155682150 wait complete. 2024-03-07T07:19:09.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.770+0000 7fbefb0dd640 1 Processor -- start 2024-03-07T07:19:09.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.770+0000 7fbefb0dd640 1 -- start start 2024-03-07T07:19:09.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbefb0dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef413a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbefb0dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4145350 0x7fbef414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbefb0dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef415c580 0x7fbef40b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbefb0dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415c9e0 con 0x7fbef4070b50 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbef8e79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef413a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbef8e79640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef413a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59690/0 (socket says 172.21.15.64:59690) 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbef8e79640 1 -- 172.21.15.64:0/1561916850 learned_addr learned my addr 172.21.15.64:0/1561916850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4145ac0 con 0x7fbef4145350 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.771+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415cb60 con 0x7fbef415c580 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef3fff640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4145350 0x7fbef414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef8e79640 1 -- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef415c580 msgr2=0x7fbef40b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef967a640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef415c580 0x7fbef40b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef8e79640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef415c580 0x7fbef40b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef8e79640 1 -- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4145350 msgr2=0x7fbef414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef8e79640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4145350 0x7fbef414e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef8e79640 1 -- 172.21.15.64:0/1561916850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef41589f0 con 0x7fbef4070b50 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef8e79640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef413a390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00e930 tx=0x7fbedc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef17fa640 1 -- 172.21.15.64:0/1561916850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc017070 con 0x7fbef4070b50 2024-03-07T07:19:09.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef17fa640 1 -- 172.21.15.64:0/1561916850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc00ca00 con 0x7fbef4070b50 2024-03-07T07:19:09.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbef17fa640 1 -- 172.21.15.64:0/1561916850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00cdf0 con 0x7fbef4070b50 2024-03-07T07:19:09.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef413d990 con 0x7fbef4070b50 2024-03-07T07:19:09.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.772+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef413e230 con 0x7fbef4070b50 2024-03-07T07:19:09.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.773+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef40d0040 con 0x7fbef4070b50 2024-03-07T07:19:09.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.774+0000 7fbef17fa640 1 -- 172.21.15.64:0/1561916850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbedc01a070 con 0x7fbef4070b50 2024-03-07T07:19:09.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.774+0000 7fbef17fa640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbed406b500 0x7fbed406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:09.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.774+0000 7fbef17fa640 1 -- 172.21.15.64:0/1561916850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbedc089f10 con 0x7fbef4070b50 2024-03-07T07:19:09.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.774+0000 7fbef3fff640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbed406b500 0x7fbed406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:09.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.775+0000 7fbef3fff640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbed406b500 0x7fbed406d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbef406f6a0 tx=0x7fbee4007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:09.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.779+0000 7fbef17fa640 1 -- 172.21.15.64:0/1561916850 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbef40d0040 con 0x7fbef4070b50 2024-03-07T07:19:09.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.961+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbef4062c90 con 0x7fbef4070b50 2024-03-07T07:19:09.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.962+0000 7fbef17fa640 1 -- 172.21.15.64:0/1561916850 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7fbedc0591a0 con 0x7fbef4070b50 2024-03-07T07:19:09.962 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:09.962 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:06.250188+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbed406b500 msgr2=0x7fbed406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbed406b500 0x7fbed406d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbef406f6a0 tx=0x7fbee4007c00 comp rx=0 tx=0).stop 2024-03-07T07:19:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 msgr2=0x7fbef413a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:09.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef413a390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00e930 tx=0x7fbedc00edf0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 shutdown_connections 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbed406b500 0x7fbed406d9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef415c580 0x7fbef40b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4145350 0x7fbef414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 --2- 172.21.15.64:0/1561916850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbef4070b50 0x7fbef413a390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 >> 172.21.15.64:0/1561916850 conn(0x7fbef406b040 msgr2=0x7fbef4070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 shutdown_connections 2024-03-07T07:19:09.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:09.964+0000 7fbefb0dd640 1 -- 172.21.15.64:0/1561916850 wait complete. 2024-03-07T07:19:09.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:09.975 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:10.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.079+0000 7fc4c6bda640 1 Processor -- start 2024-03-07T07:19:10.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.079+0000 7fc4c6bda640 1 -- start start 2024-03-07T07:19:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c6bda640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc4c0147b70 0x7fc4c0070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c6bda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c006e350 0x7fc4c00b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c6bda640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4c00b3400 0x7fc4c00714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c6bda640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c00b42e0 con 0x7fc4c00b3400 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c6bda640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c006eaa0 con 0x7fc4c0147b70 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c6bda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c006ed80 con 0x7fc4c006e350 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4c00b3400 0x7fc4c00714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc4c00b3400 0x7fc4c00714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59692/0 (socket says 172.21.15.64:59692) 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 -- 172.21.15.64:0/1127903558 learned_addr learned my addr 172.21.15.64:0/1127903558 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c4976640 1 --2- 172.21.15.64:0/1127903558 >> v2:172.21.15.120:3300/0 conn(0x7fc4c0147b70 0x7fc4c0070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4b7fff640 1 --2- 172.21.15.64:0/1127903558 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c006e350 0x7fc4c00b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 -- 172.21.15.64:0/1127903558 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c006e350 msgr2=0x7fc4c00b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 --2- 172.21.15.64:0/1127903558 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c006e350 0x7fc4c00b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 -- 172.21.15.64:0/1127903558 >> v2:172.21.15.120:3300/0 conn(0x7fc4c0147b70 msgr2=0x7fc4c0070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 --2- 172.21.15.64:0/1127903558 >> v2:172.21.15.120:3300/0 conn(0x7fc4c0147b70 0x7fc4c0070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.080+0000 7fc4c5177640 1 -- 172.21.15.64:0/1127903558 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c006f060 con 0x7fc4c00b3400 2024-03-07T07:19:10.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.081+0000 7fc4c5177640 1 --2- 172.21.15.64:0/1127903558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c00b3400 0x7fc4c00714c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0070a80 tx=0x7fc4b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b51eee592ef80a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:10.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.081+0000 7fc4b77fe640 1 -- 172.21.15.64:0/1127903558 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b802ca70 con 0x7fc4c00b3400 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.081+0000 7fc4b77fe640 1 -- 172.21.15.64:0/1127903558 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b802cc10 con 0x7fc4c00b3400 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.081+0000 7fc4b77fe640 1 -- 172.21.15.64:0/1127903558 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80343f0 con 0x7fc4c00b3400 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.081+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1127903558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c00b3400 msgr2=0x7fc4c00714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.081+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1127903558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c00b3400 0x7fc4c00714c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0070a80 tx=0x7fc4b802bed0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.082+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1127903558 shutdown_connections 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.082+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1127903558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c00b3400 0x7fc4c00714c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.082+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1127903558 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c006e350 0x7fc4c00b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.082+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1127903558 >> v2:172.21.15.120:3300/0 conn(0x7fc4c0147b70 0x7fc4c0070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.082+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1127903558 >> 172.21.15.64:0/1127903558 conn(0x7fc4c00b64d0 msgr2=0x7fc4c00b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.082+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1127903558 shutdown_connections 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.082+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1127903558 wait complete. 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.083+0000 7fc4c6bda640 1 Processor -- start 2024-03-07T07:19:10.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.083+0000 7fc4c6bda640 1 -- start start 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006e350 0x7fc4c015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c00b3400 0x7fc4c013ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0147b70 0x7fc4c0161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00ae2c0 con 0x7fc4c0147b70 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c006e770 con 0x7fc4c006e350 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00ae440 con 0x7fc4c00b3400 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0147b70 0x7fc4c0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0147b70 0x7fc4c0161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59696/0 (socket says 172.21.15.64:59696) 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 -- 172.21.15.64:0/1864254697 learned_addr learned my addr 172.21.15.64:0/1864254697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:10.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c4976640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006e350 0x7fc4c015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4b7fff640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c00b3400 0x7fc4c013ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 -- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c00b3400 msgr2=0x7fc4c013ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c00b3400 0x7fc4c013ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 -- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006e350 msgr2=0x7fc4c015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006e350 0x7fc4c015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 -- 172.21.15.64:0/1864254697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c013ca50 con 0x7fc4c0147b70 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4b7fff640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c00b3400 0x7fc4c013ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c5177640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0147b70 0x7fc4c0161060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009f40 tx=0x7fc4b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4b57fa640 1 -- 172.21.15.64:0/1864254697 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8043020 con 0x7fc4c0147b70 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4b57fa640 1 -- 172.21.15.64:0/1864254697 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b803e450 con 0x7fc4c0147b70 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c013c770 con 0x7fc4c0147b70 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4b57fa640 1 -- 172.21.15.64:0/1864254697 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8009510 con 0x7fc4c0147b70 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.084+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c015f460 con 0x7fc4c0147b70 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.085+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c0062c90 con 0x7fc4c0147b70 2024-03-07T07:19:10.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.086+0000 7fc4b57fa640 1 -- 172.21.15.64:0/1864254697 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc4b8046020 con 0x7fc4c0147b70 2024-03-07T07:19:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.086+0000 7fc4b57fa640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.086+0000 7fc4b57fa640 1 -- 172.21.15.64:0/1864254697 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4b80ac920 con 0x7fc4c0147b70 2024-03-07T07:19:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.086+0000 7fc4c4976640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.087+0000 7fc4c4976640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0099b0 tx=0x7fc4bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:10.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.090+0000 7fc4b57fa640 1 -- 172.21.15.64:0/1864254697 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc4b804d020 con 0x7fc4c0147b70 2024-03-07T07:19:10.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.271+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4c00b0740 con 0x7fc4c0147b70 2024-03-07T07:19:10.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.272+0000 7fc4b57fa640 1 -- 172.21.15.64:0/1864254697 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4b807baf0 con 0x7fc4c0147b70 2024-03-07T07:19:10.272 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc4ac06b4d0 msgr2=0x7fc4ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0099b0 tx=0x7fc4bc008040 comp rx=0 tx=0).stop 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0147b70 msgr2=0x7fc4c0161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0147b70 0x7fc4c0161060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009f40 tx=0x7fc4b8004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 shutdown_connections 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc4ac06b4d0 0x7fc4ac06d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4c0147b70 0x7fc4c0161060 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc4c00b3400 0x7fc4c013ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 --2- 172.21.15.64:0/1864254697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc4c006e350 0x7fc4c015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 >> 172.21.15.64:0/1864254697 conn(0x7fc4c00b64d0 msgr2=0x7fc4c00b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 shutdown_connections 2024-03-07T07:19:10.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.274+0000 7fc4c6bda640 1 -- 172.21.15.64:0/1864254697 wait complete. 2024-03-07T07:19:10.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.386+0000 7f0a26fee640 1 Processor -- start 2024-03-07T07:19:10.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.386+0000 7f0a26fee640 1 -- start start 2024-03-07T07:19:10.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a26fee640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b5dd0 0x7f0a20144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a26fee640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a200b43f0 0x7f0a20162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a26fee640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a200b4bd0 con 0x7f0a20162ab0 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a26fee640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a20141700 con 0x7f0a200b43f0 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a26fee640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a20161e00 con 0x7f0a200b5dd0 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a1ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a200b43f0 0x7f0a20162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a1ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0a200b43f0 0x7f0a20162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59706/0 (socket says 172.21.15.64:59706) 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2928933406 learned_addr learned my addr 172.21.15.64:0/2928933406 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a24d8a640 1 --2- 172.21.15.64:0/2928933406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b5dd0 0x7f0a20144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2928933406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b5dd0 msgr2=0x7f0a20144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a1ffff640 1 --2- 172.21.15.64:0/2928933406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b5dd0 0x7f0a20144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2928933406 >> v1:172.21.15.120:6789/0 conn(0x7f0a20162ab0 legacy=0x7f0a20071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2928933406 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200b4eb0 con 0x7f0a200b43f0 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.387+0000 7f0a24d8a640 1 --2- 172.21.15.64:0/2928933406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b5dd0 0x7f0a20144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:10.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a1ffff640 1 --2- 172.21.15.64:0/2928933406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b43f0 0x7f0a20162570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a20161f80 tx=0x7f0a1402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c06f2d7886cfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a1f7fe640 1 -- 172.21.15.64:0/2928933406 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140317c0 con 0x7f0a200b43f0 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a1f7fe640 1 -- 172.21.15.64:0/2928933406 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14037070 con 0x7f0a200b43f0 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a1f7fe640 1 -- 172.21.15.64:0/2928933406 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14031b80 con 0x7f0a200b43f0 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a26fee640 1 -- 172.21.15.64:0/2928933406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b43f0 msgr2=0x7f0a20162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2928933406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b43f0 0x7f0a20162570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a20161f80 tx=0x7f0a1402b860 comp rx=0 tx=0).stop 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a26fee640 1 -- 172.21.15.64:0/2928933406 shutdown_connections 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2928933406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b43f0 0x7f0a20162570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2928933406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b5dd0 0x7f0a20144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.388+0000 7f0a26fee640 1 -- 172.21.15.64:0/2928933406 >> 172.21.15.64:0/2928933406 conn(0x7f0a200d0e70 msgr2=0x7f0a200d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.389+0000 7f0a26fee640 1 -- 172.21.15.64:0/2928933406 shutdown_connections 2024-03-07T07:19:10.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.389+0000 7f0a26fee640 1 -- 172.21.15.64:0/2928933406 wait complete. 2024-03-07T07:19:10.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.390+0000 7f0a26fee640 1 Processor -- start 2024-03-07T07:19:10.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.390+0000 7f0a26fee640 1 -- start start 2024-03-07T07:19:10.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b43f0 0x7f0a20143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b5dd0 0x7f0a201473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a20142d00 0x7f0a2015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200b61b0 con 0x7f0a200b5dd0 2024-03-07T07:19:10.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a20162050 con 0x7f0a20142d00 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200b4870 con 0x7f0a200b43f0 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b5dd0 0x7f0a201473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b5dd0 0x7f0a201473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59716/0 (socket says 172.21.15.64:59716) 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2324075517 learned_addr learned my addr 172.21.15.64:0/2324075517 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a24d8a640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b43f0 0x7f0a20143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a2558b640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a20142d00 0x7f0a2015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b43f0 msgr2=0x7f0a20143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b43f0 0x7f0a20143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a20142d00 msgr2=0x7f0a2015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a20142d00 0x7f0a2015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 -- 172.21.15.64:0/2324075517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20143a60 con 0x7f0a200b5dd0 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a2558b640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a20142d00 0x7f0a2015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:10.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1ffff640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b5dd0 0x7f0a201473b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a14000c00 tx=0x7f0a140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1d7fa640 1 -- 172.21.15.64:0/2324075517 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14032860 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1d7fa640 1 -- 172.21.15.64:0/2324075517 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14032a00 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a1d7fa640 1 -- 172.21.15.64:0/2324075517 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14032dd0 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a20141c60 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.391+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a20142220 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.392+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a20062c90 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.393+0000 7f0a1d7fa640 1 -- 172.21.15.64:0/2324075517 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0a1404c060 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.393+0000 7f0a1d7fa640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0a0c06b4d0 0x7f0a0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.393+0000 7f0a1d7fa640 1 -- 172.21.15.64:0/2324075517 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0a140abb60 con 0x7f0a200b5dd0 2024-03-07T07:19:10.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.393+0000 7f0a24d8a640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0a0c06b4d0 0x7f0a0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:10.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.394+0000 7f0a24d8a640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0a0c06b4d0 0x7f0a0c06d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d620 tx=0x7f0a18007810 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:10.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.397+0000 7f0a1d7fa640 1 -- 172.21.15.64:0/2324075517 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0a14040070 con 0x7f0a200b5dd0 2024-03-07T07:19:10.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.617+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a201478f0 con 0x7f0a200b5dd0 2024-03-07T07:19:10.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.617+0000 7f0a1d7fa640 1 -- 172.21.15.64:0/2324075517 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f0a1407ad30 con 0x7f0a200b5dd0 2024-03-07T07:19:10.617 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:10.618 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-07T07:19:10.618 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0a0c06b4d0 msgr2=0x7f0a0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0a0c06b4d0 0x7f0a0c06d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d620 tx=0x7f0a18007810 comp rx=0 tx=0).stop 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b5dd0 msgr2=0x7f0a201473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b5dd0 0x7f0a201473b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a14000c00 tx=0x7f0a140046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 shutdown_connections 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0a0c06b4d0 0x7f0a0c06d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a200b5dd0 0x7f0a201473b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0a20142d00 0x7f0a2015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 --2- 172.21.15.64:0/2324075517 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0a200b43f0 0x7f0a20143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:10.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 >> 172.21.15.64:0/2324075517 conn(0x7f0a200d0e70 msgr2=0x7f0a200d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:10.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 shutdown_connections 2024-03-07T07:19:10.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:10.619+0000 7f0a26fee640 1 -- 172.21.15.64:0/2324075517 wait complete. 2024-03-07T07:19:12.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:12.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.745+0000 7fd760599640 1 Processor -- start 2024-03-07T07:19:12.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.745+0000 7fd760599640 1 -- start start 2024-03-07T07:19:12.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd760599640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd75806def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:12.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd760599640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd758070640 0x7fd758070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd760599640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75815ea50 con 0x7fd758155440 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd760599640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75806e430 con 0x7fd758070640 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd760599640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75815ed30 con 0x7fd75806dab0 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75e335640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd758155440 0x7fd758066eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37362/0 (socket says 172.21.15.64:37362) 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75e335640 1 -- 172.21.15.64:0/3929142412 learned_addr learned my addr 172.21.15.64:0/3929142412 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75db34640 1 --2- 172.21.15.64:0/3929142412 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd75806def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75eb36640 1 --2- 172.21.15.64:0/3929142412 >> v2:172.21.15.120:3300/0 conn(0x7fd758070640 0x7fd758070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75d333640 1 -- 172.21.15.64:0/3929142412 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905575435 0 0) 0x7fd75815ea50 con 0x7fd758155440 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75d333640 1 -- 172.21.15.64:0/3929142412 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd744003520 con 0x7fd758155440 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75d333640 1 -- 172.21.15.64:0/3929142412 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413954041 0 0) 0x7fd744003520 con 0x7fd758155440 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75db34640 1 -- 172.21.15.64:0/3929142412 >> v2:172.21.15.120:3300/0 conn(0x7fd758070640 msgr2=0x7fd758070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:12.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75db34640 1 --2- 172.21.15.64:0/3929142412 >> v2:172.21.15.120:3300/0 conn(0x7fd758070640 0x7fd758070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75db34640 1 -- 172.21.15.64:0/3929142412 >> v1:172.21.15.64:6789/0 conn(0x7fd758155440 legacy=0x7fd758066eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:12.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75db34640 1 -- 172.21.15.64:0/3929142412 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75815f010 con 0x7fd75806dab0 2024-03-07T07:19:12.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.746+0000 7fd75eb36640 1 --2- 172.21.15.64:0/3929142412 >> v2:172.21.15.120:3300/0 conn(0x7fd758070640 0x7fd758070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:12.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.747+0000 7fd75db34640 1 --2- 172.21.15.64:0/3929142412 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd75806def0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd7480097a0 tx=0x7fd74802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9912abf0cd663ad server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:12.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.747+0000 7fd75d333640 1 -- 172.21.15.64:0/3929142412 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748037070 con 0x7fd75806dab0 2024-03-07T07:19:12.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.747+0000 7fd75d333640 1 -- 172.21.15.64:0/3929142412 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748030510 con 0x7fd75806dab0 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.747+0000 7fd75d333640 1 -- 172.21.15.64:0/3929142412 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754001670 con 0x7fd75806dab0 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 -- 172.21.15.64:0/3929142412 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 msgr2=0x7fd75806def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 --2- 172.21.15.64:0/3929142412 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd75806def0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd7480097a0 tx=0x7fd74802b790 comp rx=0 tx=0).stop 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 -- 172.21.15.64:0/3929142412 shutdown_connections 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 --2- 172.21.15.64:0/3929142412 >> v2:172.21.15.120:3300/0 conn(0x7fd758070640 0x7fd758070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 --2- 172.21.15.64:0/3929142412 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd75806def0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 -- 172.21.15.64:0/3929142412 >> 172.21.15.64:0/3929142412 conn(0x7fd7580cd160 msgr2=0x7fd7580cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 -- 172.21.15.64:0/3929142412 shutdown_connections 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.748+0000 7fd760599640 1 -- 172.21.15.64:0/3929142412 wait complete. 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.749+0000 7fd760599640 1 Processor -- start 2024-03-07T07:19:12.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.749+0000 7fd760599640 1 -- start start 2024-03-07T07:19:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd760599640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd7581548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd760599640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd758070640 0x7fd758152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd760599640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758155440 0x7fd758138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd760599640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75815e740 con 0x7fd758155440 2024-03-07T07:19:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd760599640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75815f5d0 con 0x7fd758070640 2024-03-07T07:19:12.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd760599640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75806f1a0 con 0x7fd75806dab0 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758155440 0x7fd758138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758155440 0x7fd758138850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40560/0 (socket says 172.21.15.64:40560) 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 -- 172.21.15.64:0/4037683152 learned_addr learned my addr 172.21.15.64:0/4037683152 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75db34640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd758070640 0x7fd758152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75e335640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd7581548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 -- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 msgr2=0x7fd7581548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd7581548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 -- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd758070640 msgr2=0x7fd758152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd758070640 0x7fd758152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.750+0000 7fd75eb36640 1 -- 172.21.15.64:0/4037683152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758152f70 con 0x7fd758155440 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.751+0000 7fd75eb36640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758155440 0x7fd758138850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd75000efc0 tx=0x7fd75000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.751+0000 7fd74effd640 1 -- 172.21.15.64:0/4037683152 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750018030 con 0x7fd758155440 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.751+0000 7fd74effd640 1 -- 172.21.15.64:0/4037683152 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75000cd30 con 0x7fd758155440 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.751+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758153210 con 0x7fd758155440 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.751+0000 7fd74effd640 1 -- 172.21.15.64:0/4037683152 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750003bb0 con 0x7fd758155440 2024-03-07T07:19:12.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.751+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7581537d0 con 0x7fd758155440 2024-03-07T07:19:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.752+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758162a20 con 0x7fd758155440 2024-03-07T07:19:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.752+0000 7fd74effd640 1 -- 172.21.15.64:0/4037683152 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd750005d10 con 0x7fd758155440 2024-03-07T07:19:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.753+0000 7fd74effd640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd73c06b4d0 0x7fd73c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.753+0000 7fd74effd640 1 -- 172.21.15.64:0/4037683152 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd75008a7a0 con 0x7fd758155440 2024-03-07T07:19:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.753+0000 7fd75e335640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd73c06b4d0 0x7fd73c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:12.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.753+0000 7fd75e335640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd73c06b4d0 0x7fd73c06d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd754001320 tx=0x7fd7540012b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:12.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.757+0000 7fd74effd640 1 -- 172.21.15.64:0/4037683152 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd758162a20 con 0x7fd758155440 2024-03-07T07:19:12.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.976+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd758062c90 con 0x7fd758155440 2024-03-07T07:19:12.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.976+0000 7fd74effd640 1 -- 172.21.15.64:0/4037683152 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7fd750022070 con 0x7fd758155440 2024-03-07T07:19:12.977 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:12.977 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-07T07:19:12.977 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2024-03-07T07:19:12.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd73c06b4d0 msgr2=0x7fd73c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd73c06b4d0 0x7fd73c06d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd754001320 tx=0x7fd7540012b0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758155440 msgr2=0x7fd758138850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758155440 0x7fd758138850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd75000efc0 tx=0x7fd75000c490 comp rx=0 tx=0).stop 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 shutdown_connections 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd73c06b4d0 0x7fd73c06d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd758155440 0x7fd758138850 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd758070640 0x7fd758152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 --2- 172.21.15.64:0/4037683152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd75806dab0 0x7fd7581548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 >> 172.21.15.64:0/4037683152 conn(0x7fd7580cd160 msgr2=0x7fd758066eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 shutdown_connections 2024-03-07T07:19:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:12.978+0000 7fd760599640 1 -- 172.21.15.64:0/4037683152 wait complete. 2024-03-07T07:19:12.989 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:13.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.090+0000 7fbb42db8640 1 Processor -- start 2024-03-07T07:19:13.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.090+0000 7fbb42db8640 1 -- start start 2024-03-07T07:19:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb42db8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb3c145bf0 0x7fbb3c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb42db8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbb3c0d1860 0x7fbb3c06b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb42db8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c0d0aa0 con 0x7fbb3c0b7390 2024-03-07T07:19:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb42db8640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c0d13d0 con 0x7fbb3c145bf0 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb42db8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c0d0d80 con 0x7fbb3c0d1860 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb40b54640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb3c145bf0 0x7fbb3c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb40b54640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fbb3c145bf0 0x7fbb3c0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40562/0 (socket says 172.21.15.64:40562) 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.091+0000 7fbb40b54640 1 -- 172.21.15.64:0/3961825871 learned_addr learned my addr 172.21.15.64:0/3961825871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.092+0000 7fbb40b54640 1 -- 172.21.15.64:0/3961825871 >> v2:172.21.15.120:3300/0 conn(0x7fbb3c0d1860 msgr2=0x7fbb3c06b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.092+0000 7fbb3bfff640 1 --2- 172.21.15.64:0/3961825871 >> v2:172.21.15.120:3300/0 conn(0x7fbb3c0d1860 0x7fbb3c06b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.092+0000 7fbb40b54640 1 --2- 172.21.15.64:0/3961825871 >> v2:172.21.15.120:3300/0 conn(0x7fbb3c0d1860 0x7fbb3c06b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.092+0000 7fbb40b54640 1 -- 172.21.15.64:0/3961825871 >> v1:172.21.15.120:6789/0 conn(0x7fbb3c0b7390 legacy=0x7fbb3c15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.092+0000 7fbb40b54640 1 -- 172.21.15.64:0/3961825871 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0d1060 con 0x7fbb3c145bf0 2024-03-07T07:19:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.092+0000 7fbb3bfff640 1 --2- 172.21.15.64:0/3961825871 >> v2:172.21.15.120:3300/0 conn(0x7fbb3c0d1860 0x7fbb3c06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:13.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.092+0000 7fbb40b54640 1 --2- 172.21.15.64:0/3961825871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c145bf0 0x7fbb3c0cea70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c06c130 tx=0x7fbb3002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2349f11d0c5df6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb3b7fe640 1 -- 172.21.15.64:0/3961825871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30035020 con 0x7fbb3c145bf0 2024-03-07T07:19:13.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb3b7fe640 1 -- 172.21.15.64:0/3961825871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb30031640 con 0x7fbb3c145bf0 2024-03-07T07:19:13.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb3b7fe640 1 -- 172.21.15.64:0/3961825871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3002d040 con 0x7fbb3c145bf0 2024-03-07T07:19:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 -- 172.21.15.64:0/3961825871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c145bf0 msgr2=0x7fbb3c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3961825871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c145bf0 0x7fbb3c0cea70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c06c130 tx=0x7fbb3002b8f0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 -- 172.21.15.64:0/3961825871 shutdown_connections 2024-03-07T07:19:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3961825871 >> v2:172.21.15.120:3300/0 conn(0x7fbb3c0d1860 0x7fbb3c06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3961825871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c145bf0 0x7fbb3c0cea70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 -- 172.21.15.64:0/3961825871 >> 172.21.15.64:0/3961825871 conn(0x7fbb3c06d4c0 msgr2=0x7fbb3c066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 -- 172.21.15.64:0/3961825871 shutdown_connections 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.093+0000 7fbb42db8640 1 -- 172.21.15.64:0/3961825871 wait complete. 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.094+0000 7fbb42db8640 1 Processor -- start 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.094+0000 7fbb42db8640 1 -- start start 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb42db8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c0b7390 0x7fbb3c15b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb42db8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb3c0d1860 0x7fbb3c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb42db8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb3c145bf0 0x7fbb3c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb42db8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0cf130 con 0x7fbb3c0b7390 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb42db8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0cefb0 con 0x7fbb3c145bf0 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb42db8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0d06c0 con 0x7fbb3c0d1860 2024-03-07T07:19:13.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb40b54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c0b7390 0x7fbb3c15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb40b54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c0b7390 0x7fbb3c15b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40578/0 (socket says 172.21.15.64:40578) 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb40b54640 1 -- 172.21.15.64:0/3683260572 learned_addr learned my addr 172.21.15.64:0/3683260572 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.095+0000 7fbb41355640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb3c145bf0 0x7fbb3c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb3bfff640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb3c0d1860 0x7fbb3c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb40b54640 1 -- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb3c0d1860 msgr2=0x7fbb3c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb40b54640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb3c0d1860 0x7fbb3c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb40b54640 1 -- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb3c145bf0 msgr2=0x7fbb3c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb40b54640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb3c145bf0 0x7fbb3c0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb40b54640 1 -- 172.21.15.64:0/3683260572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c15bf40 con 0x7fbb3c0b7390 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb3bfff640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb3c0d1860 0x7fbb3c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb40b54640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c0b7390 0x7fbb3c15b610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb30000c00 tx=0x7fbb300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb397fa640 1 -- 172.21.15.64:0/3683260572 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30040070 con 0x7fbb3c0b7390 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb397fa640 1 -- 172.21.15.64:0/3683260572 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb30039c10 con 0x7fbb3c0b7390 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c0cb710 con 0x7fbb3c0b7390 2024-03-07T07:19:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb397fa640 1 -- 172.21.15.64:0/3683260572 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30035040 con 0x7fbb3c0b7390 2024-03-07T07:19:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.096+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c0cbcd0 con 0x7fbb3c0b7390 2024-03-07T07:19:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.097+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c062c90 con 0x7fbb3c0b7390 2024-03-07T07:19:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.097+0000 7fbb397fa640 1 -- 172.21.15.64:0/3683260572 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbb3004b060 con 0x7fbb3c0b7390 2024-03-07T07:19:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.098+0000 7fbb397fa640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb2006b4d0 0x7fbb2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.098+0000 7fbb397fa640 1 -- 172.21.15.64:0/3683260572 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb300ab960 con 0x7fbb3c0b7390 2024-03-07T07:19:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.098+0000 7fbb3bfff640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb2006b4d0 0x7fbb2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.098+0000 7fbb3bfff640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb2006b4d0 0x7fbb2006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006fd0 tx=0x7fbb28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.102+0000 7fbb397fa640 1 -- 172.21.15.64:0/3683260572 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb3c15bf40 con 0x7fbb3c0b7390 2024-03-07T07:19:13.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.285+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb3c0b4b60 con 0x7fbb3c0b7390 2024-03-07T07:19:13.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.285+0000 7fbb397fa640 1 -- 172.21.15.64:0/3683260572 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7fbb3002f300 con 0x7fbb3c0b7390 2024-03-07T07:19:13.285 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:13.285 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":16,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:06.250188+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.287+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb2006b4d0 msgr2=0x7fbb2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.287+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb2006b4d0 0x7fbb2006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb28006fd0 tx=0x7fbb28008040 comp rx=0 tx=0).stop 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.287+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c0b7390 msgr2=0x7fbb3c15b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.287+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c0b7390 0x7fbb3c15b610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb30000c00 tx=0x7fbb300046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 shutdown_connections 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb2006b4d0 0x7fbb2006d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb3c145bf0 0x7fbb3c0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb3c0d1860 0x7fbb3c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 --2- 172.21.15.64:0/3683260572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c0b7390 0x7fbb3c15b610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 >> 172.21.15.64:0/3683260572 conn(0x7fbb3c06d4c0 msgr2=0x7fbb3c06b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:13.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 shutdown_connections 2024-03-07T07:19:13.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.288+0000 7fbb42db8640 1 -- 172.21.15.64:0/3683260572 wait complete. 2024-03-07T07:19:13.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:13.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:13.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.403+0000 7f1964287640 1 Processor -- start 2024-03-07T07:19:13.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.403+0000 7f1964287640 1 -- start start 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1964287640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f195c15e870 0x7f195c0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1964287640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195c070eb0 con 0x7f195c0cea70 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1964287640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195c071190 con 0x7f195c0d3310 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1964287640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c071470 con 0x7f195c15e870 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961822640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f195c0cea70 0x7f195c15ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37368/0 (socket says 172.21.15.64:37368) 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961822640 1 -- 172.21.15.64:0/462818300 learned_addr learned my addr 172.21.15.64:0/462818300 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1962023640 1 --2- 172.21.15.64:0/462818300 >> v2:172.21.15.120:3300/0 conn(0x7f195c15e870 0x7f195c0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906128537 0 0) 0x7f195c070eb0 con 0x7f195c0cea70 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1948003520 con 0x7f195c0cea70 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 197557115 0 0) 0x7f1948003520 con 0x7f195c0cea70 2024-03-07T07:19:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 >> v2:172.21.15.120:3300/0 conn(0x7f195c15e870 msgr2=0x7f195c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 --2- 172.21.15.64:0/462818300 >> v2:172.21.15.120:3300/0 conn(0x7f195c15e870 0x7f195c0cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 >> v1:172.21.15.120:6789/0 conn(0x7f195c0d3310 legacy=0x7f195c167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c071750 con 0x7f195c0cea70 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f194c001670 con 0x7f195c0cea70 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 >> v1:172.21.15.64:6789/0 conn(0x7f195c0cea70 legacy=0x7f195c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19480039b0 0x7f1948003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1948004800 0x7f1948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480051c0 0x7f194800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c071470 con 0x7f19480039b0 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194800c060 con 0x7f19480051c0 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194800c260 con 0x7f1948004800 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.404+0000 7f1962023640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19480039b0 0x7f1948003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962824640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480051c0 0x7f194800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1961822640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1948004800 0x7f1948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962023640 1 -- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1948004800 msgr2=0x7f1948004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962023640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1948004800 0x7f1948004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962023640 1 -- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480051c0 msgr2=0x7f194800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962023640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480051c0 0x7f194800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962824640 1 -- 172.21.15.64:0/462818300 reap_dead start 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962023640 1 -- 172.21.15.64:0/462818300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1948003520 con 0x7f19480039b0 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1962023640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19480039b0 0x7f1948003dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ac0 tx=0x7f1948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e144e6305fb8d314 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.405+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958039070 con 0x7f19480039b0 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195802c6b0 con 0x7f19480039b0 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1961021640 1 -- 172.21.15.64:0/462818300 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958009690 con 0x7f19480039b0 2024-03-07T07:19:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1964287640 1 -- 172.21.15.64:0/462818300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19480039b0 msgr2=0x7f1948003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1964287640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19480039b0 0x7f1948003dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ac0 tx=0x7f1948005830 comp rx=0 tx=0).stop 2024-03-07T07:19:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1964287640 1 -- 172.21.15.64:0/462818300 shutdown_connections 2024-03-07T07:19:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1964287640 1 --2- 172.21.15.64:0/462818300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19480039b0 0x7f1948003dd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1964287640 1 -- 172.21.15.64:0/462818300 >> 172.21.15.64:0/462818300 conn(0x7f195c0d11e0 msgr2=0x7f195c0d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1964287640 1 -- 172.21.15.64:0/462818300 shutdown_connections 2024-03-07T07:19:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.406+0000 7f1964287640 1 -- 172.21.15.64:0/462818300 wait complete. 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.407+0000 7f1964287640 1 Processor -- start 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.407+0000 7f1964287640 1 -- start start 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1964287640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480039b0 0x7f195c145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1964287640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c14e610 0x7f195c1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1964287640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c151a10 0x7f195c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1964287640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0cf190 con 0x7f195c14e610 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1964287640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c070b60 con 0x7f19480039b0 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1964287640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0cf310 con 0x7f195c151a10 2024-03-07T07:19:13.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c14e610 0x7f195c1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c14e610 0x7f195c1514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40600/0 (socket says 172.21.15.64:40600) 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 -- 172.21.15.64:0/286054115 learned_addr learned my addr 172.21.15.64:0/286054115 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1962023640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480039b0 0x7f195c145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 -- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c151a10 msgr2=0x7f195c153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1962824640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c151a10 0x7f195c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c151a10 0x7f195c153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 -- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480039b0 msgr2=0x7f195c145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480039b0 0x7f195c145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1961822640 1 -- 172.21.15.64:0/286054115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c161000 con 0x7f195c14e610 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.408+0000 7f1962023640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480039b0 0x7f195c145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.409+0000 7f1961822640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c14e610 0x7f195c1514d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f194c00e7f0 tx=0x7f194c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.409+0000 7f1952ffd640 1 -- 172.21.15.64:0/286054115 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00c870 con 0x7f195c14e610 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.409+0000 7f1952ffd640 1 -- 172.21.15.64:0/286054115 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c018070 con 0x7f195c14e610 2024-03-07T07:19:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.409+0000 7f1952ffd640 1 -- 172.21.15.64:0/286054115 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00cc30 con 0x7f195c14e610 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.409+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c152880 con 0x7f195c14e610 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.409+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c152e40 con 0x7f195c14e610 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.410+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c062c90 con 0x7f195c14e610 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.410+0000 7f1952ffd640 1 -- 172.21.15.64:0/286054115 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f194c013420 con 0x7f195c14e610 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.411+0000 7f1952ffd640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f194006b500 0x7f194006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.411+0000 7f1952ffd640 1 -- 172.21.15.64:0/286054115 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f194c089a90 con 0x7f195c14e610 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.411+0000 7f1962023640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f194006b500 0x7f194006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.411+0000 7f1962023640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f194006b500 0x7f194006d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1958036000 tx=0x7f1958030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.415+0000 7f1952ffd640 1 -- 172.21.15.64:0/286054115 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f194c058bd0 con 0x7f195c14e610 2024-03-07T07:19:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.595+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f195c0d2bd0 con 0x7f195c14e610 2024-03-07T07:19:13.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.595+0000 7f1952ffd640 1 -- 172.21.15.64:0/286054115 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f194c0584d0 con 0x7f195c14e610 2024-03-07T07:19:13.596 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f194006b500 msgr2=0x7f194006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f194006b500 0x7f194006d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1958036000 tx=0x7f1958030000 comp rx=0 tx=0).stop 2024-03-07T07:19:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c14e610 msgr2=0x7f195c1514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c14e610 0x7f195c1514d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f194c00e7f0 tx=0x7f194c00ecb0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 shutdown_connections 2024-03-07T07:19:13.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f194006b500 0x7f194006d9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f195c14e610 0x7f195c1514d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f195c151a10 0x7f195c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 --2- 172.21.15.64:0/286054115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19480039b0 0x7f195c145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 >> 172.21.15.64:0/286054115 conn(0x7f195c0d11e0 msgr2=0x7f195c0b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:13.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 shutdown_connections 2024-03-07T07:19:13.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.597+0000 7f1964287640 1 -- 172.21.15.64:0/286054115 wait complete. 2024-03-07T07:19:13.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-07T07:19:13.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.709+0000 7f788fdb8640 1 Processor -- start 2024-03-07T07:19:13.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.710+0000 7f788fdb8640 1 -- start start 2024-03-07T07:19:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.710+0000 7f788fdb8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f788815e0d0 0x7f78881417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.710+0000 7f788fdb8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78880b5cd0 0x7f7888147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.710+0000 7f788fdb8640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78880b4520 con 0x7f7888141d10 2024-03-07T07:19:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.710+0000 7f788fdb8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78880b4800 con 0x7f788815e0d0 2024-03-07T07:19:13.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.710+0000 7f788fdb8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78880b4ae0 con 0x7f78880b5cd0 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788d353640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7888141d10 0x7f78880b55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37380/0 (socket says 172.21.15.64:37380) 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788d353640 1 -- 172.21.15.64:0/2274005495 learned_addr learned my addr 172.21.15.64:0/2274005495 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788db54640 1 --2- 172.21.15.64:0/2274005495 >> v2:172.21.15.120:3300/0 conn(0x7f788815e0d0 0x7f78881417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788e355640 1 --2- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78880b5cd0 0x7f7888147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788cb52640 1 -- 172.21.15.64:0/2274005495 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228296307 0 0) 0x7f78880b4520 con 0x7f7888141d10 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788cb52640 1 -- 172.21.15.64:0/2274005495 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7874003520 con 0x7f7888141d10 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788db54640 1 -- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78880b5cd0 msgr2=0x7f7888147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788db54640 1 --2- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78880b5cd0 0x7f7888147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788db54640 1 -- 172.21.15.64:0/2274005495 >> v1:172.21.15.64:6789/0 conn(0x7f7888141d10 legacy=0x7f78880b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788db54640 1 -- 172.21.15.64:0/2274005495 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880b4dc0 con 0x7f788815e0d0 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.711+0000 7f788e355640 1 --2- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78880b5cd0 0x7f7888147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:13.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788db54640 1 --2- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f788815e0d0 0x7f78881417d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78840097a0 tx=0x7f788402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5120e6bd49cda44 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788cb52640 1 -- 172.21.15.64:0/2274005495 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884039070 con 0x7f788815e0d0 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788cb52640 1 -- 172.21.15.64:0/2274005495 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78840316f0 con 0x7f788815e0d0 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788cb52640 1 -- 172.21.15.64:0/2274005495 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884031aa0 con 0x7f788815e0d0 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788fdb8640 1 -- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f788815e0d0 msgr2=0x7f78881417d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788fdb8640 1 --2- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f788815e0d0 0x7f78881417d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78840097a0 tx=0x7f788402b790 comp rx=0 tx=0).stop 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788fdb8640 1 -- 172.21.15.64:0/2274005495 shutdown_connections 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788fdb8640 1 --2- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78880b5cd0 0x7f7888147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.712+0000 7f788fdb8640 1 --2- 172.21.15.64:0/2274005495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f788815e0d0 0x7f78881417d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.713+0000 7f788fdb8640 1 -- 172.21.15.64:0/2274005495 >> 172.21.15.64:0/2274005495 conn(0x7f7888071300 msgr2=0x7f7888071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.713+0000 7f788fdb8640 1 -- 172.21.15.64:0/2274005495 shutdown_connections 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.713+0000 7f788fdb8640 1 -- 172.21.15.64:0/2274005495 wait complete. 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 Processor -- start 2024-03-07T07:19:13.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 -- start start 2024-03-07T07:19:13.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78880b5cd0 0x7f78881f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888141d10 0x7f788815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f788815e0d0 0x7f788815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78881475c0 con 0x7f78880b5cd0 2024-03-07T07:19:13.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788815e4b0 con 0x7f7888141d10 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.714+0000 7f788fdb8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7888147740 con 0x7f788815e0d0 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78880b5cd0 0x7f78881f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78880b5cd0 0x7f78881f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40610/0 (socket says 172.21.15.64:40610) 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 -- 172.21.15.64:0/567830696 learned_addr learned my addr 172.21.15.64:0/567830696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 -- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f788815e0d0 msgr2=0x7f788815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f788815e0d0 0x7f788815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 -- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888141d10 msgr2=0x7f788815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788d353640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888141d10 0x7f788815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888141d10 0x7f788815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 -- 172.21.15.64:0/567830696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78881f3e90 con 0x7f78880b5cd0 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788db54640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78880b5cd0 0x7f78881f3950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7884037040 tx=0x7f78840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f787e7fc640 1 -- 172.21.15.64:0/567830696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884039040 con 0x7f78880b5cd0 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7888144d80 con 0x7f78880b5cd0 2024-03-07T07:19:13.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7888145340 con 0x7f78880b5cd0 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f787e7fc640 1 -- 172.21.15.64:0/567830696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7884031e60 con 0x7f78880b5cd0 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.715+0000 7f787e7fc640 1 -- 172.21.15.64:0/567830696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884004230 con 0x7f78880b5cd0 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.716+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7888062c90 con 0x7f78880b5cd0 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.716+0000 7f787e7fc640 1 -- 172.21.15.64:0/567830696 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f788403c070 con 0x7f78880b5cd0 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.717+0000 7f787e7fc640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f786c06b4d0 0x7f786c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.717+0000 7f787e7fc640 1 -- 172.21.15.64:0/567830696 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f78840ab410 con 0x7f78880b5cd0 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.717+0000 7f788d353640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f786c06b4d0 0x7f786c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:13.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.717+0000 7f788d353640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f786c06b4d0 0x7f786c06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7888130090 tx=0x7f7878009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:13.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.721+0000 7f787e7fc640 1 -- 172.21.15.64:0/567830696 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f788404c030 con 0x7f78880b5cd0 2024-03-07T07:19:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.906+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7888161a20 con 0x7f78880b5cd0 2024-03-07T07:19:13.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.906+0000 7f787e7fc640 1 -- 172.21.15.64:0/567830696 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7884043030 con 0x7f78880b5cd0 2024-03-07T07:19:13.907 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:13.907 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-03-07T07:19:13.907 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:19:13.907 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-03-07T07:19:13.908 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-03-07T07:19:13.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:19:13.908 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-03-07T07:19:13.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f786c06b4d0 msgr2=0x7f786c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f786c06b4d0 0x7f786c06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7888130090 tx=0x7f7878009040 comp rx=0 tx=0).stop 2024-03-07T07:19:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78880b5cd0 msgr2=0x7f78881f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78880b5cd0 0x7f78881f3950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7884037040 tx=0x7f78840046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 shutdown_connections 2024-03-07T07:19:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f786c06b4d0 0x7f786c06d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f788815e0d0 0x7f788815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7888141d10 0x7f788815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 --2- 172.21.15.64:0/567830696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78880b5cd0 0x7f78881f3950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 >> 172.21.15.64:0/567830696 conn(0x7f7888071300 msgr2=0x7f78880d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 shutdown_connections 2024-03-07T07:19:13.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:13.908+0000 7f788fdb8640 1 -- 172.21.15.64:0/567830696 wait complete. 2024-03-07T07:19:13.919 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-07T07:19:13.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-07T07:19:14.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.024+0000 7f0fed8a5640 1 Processor -- start 2024-03-07T07:19:14.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.024+0000 7f0fed8a5640 1 -- start start 2024-03-07T07:19:14.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fed8a5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0fe8145920 0x7f0fe80b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fed8a5640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0fe806bc20 0x7f0fe80d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fed8a5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe80b5490 con 0x7f0fe80b4e70 2024-03-07T07:19:14.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fed8a5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe80b60a0 con 0x7f0fe8145920 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fed8a5640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe80b5770 con 0x7f0fe806bc20 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fe6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0fe8145920 0x7f0fe80b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fe6ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0fe8145920 0x7f0fe80b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40612/0 (socket says 172.21.15.64:40612) 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fe6ffd640 1 -- 172.21.15.64:0/3953924854 learned_addr learned my addr 172.21.15.64:0/3953924854 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.025+0000 7f0fe67fc640 1 --1- 172.21.15.64:0/3953924854 >> v1:172.21.15.64:6789/0 conn(0x7f0fe80b4e70 0x7f0fe806b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37388/0 (socket says 172.21.15.64:37388) 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.026+0000 7f0fe77fe640 1 --2- 172.21.15.64:0/3953924854 >> v2:172.21.15.120:3300/0 conn(0x7f0fe806bc20 0x7f0fe80d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.026+0000 7f0fe6ffd640 1 -- 172.21.15.64:0/3953924854 >> v2:172.21.15.120:3300/0 conn(0x7f0fe806bc20 msgr2=0x7f0fe80d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.026+0000 7f0fe6ffd640 1 --2- 172.21.15.64:0/3953924854 >> v2:172.21.15.120:3300/0 conn(0x7f0fe806bc20 0x7f0fe80d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.026+0000 7f0fe6ffd640 1 -- 172.21.15.64:0/3953924854 >> v1:172.21.15.64:6789/0 conn(0x7f0fe80b4e70 legacy=0x7f0fe806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.026+0000 7f0fe6ffd640 1 -- 172.21.15.64:0/3953924854 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80b5a50 con 0x7f0fe8145920 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.026+0000 7f0fe6ffd640 1 --2- 172.21.15.64:0/3953924854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe8145920 0x7f0fe80b6c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80b6010 tx=0x7f0fe002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b973a7ebccbe5a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.026+0000 7f0fe5ffb640 1 -- 172.21.15.64:0/3953924854 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0035020 con 0x7f0fe8145920 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fe5ffb640 1 -- 172.21.15.64:0/3953924854 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe0031870 con 0x7f0fe8145920 2024-03-07T07:19:14.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fe5ffb640 1 -- 172.21.15.64:0/3953924854 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0031c40 con 0x7f0fe8145920 2024-03-07T07:19:14.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 -- 172.21.15.64:0/3953924854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe8145920 msgr2=0x7f0fe80b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/3953924854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe8145920 0x7f0fe80b6c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80b6010 tx=0x7f0fe002b910 comp rx=0 tx=0).stop 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 -- 172.21.15.64:0/3953924854 shutdown_connections 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/3953924854 >> v2:172.21.15.120:3300/0 conn(0x7f0fe806bc20 0x7f0fe80d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/3953924854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe8145920 0x7f0fe80b6c30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 -- 172.21.15.64:0/3953924854 >> 172.21.15.64:0/3953924854 conn(0x7f0fe8066e50 msgr2=0x7f0fe8067260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 -- 172.21.15.64:0/3953924854 shutdown_connections 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.027+0000 7f0fed8a5640 1 -- 172.21.15.64:0/3953924854 wait complete. 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.028+0000 7f0fed8a5640 1 Processor -- start 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.028+0000 7f0fed8a5640 1 -- start start 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fed8a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fe806bc20 0x7f0fe8159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fed8a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe80b4e70 0x7f0fe812d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fed8a5640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fe8145920 0x7f0fe81e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fed8a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80b7170 con 0x7f0fe80b4e70 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fed8a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe8145d00 con 0x7f0fe806bc20 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fed8a5640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80d0a10 con 0x7f0fe8145920 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fe67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe80b4e70 0x7f0fe812d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fe67fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe80b4e70 0x7f0fe812d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40620/0 (socket says 172.21.15.64:40620) 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fe67fc640 1 -- 172.21.15.64:0/2045225758 learned_addr learned my addr 172.21.15.64:0/2045225758 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fe6ffd640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fe806bc20 0x7f0fe8159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.029+0000 7f0fe77fe640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fe8145920 0x7f0fe81e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fe67fc640 1 -- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fe8145920 msgr2=0x7f0fe81e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fe67fc640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fe8145920 0x7f0fe81e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fe67fc640 1 -- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fe806bc20 msgr2=0x7f0fe8159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fe67fc640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fe806bc20 0x7f0fe8159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fe67fc640 1 -- 172.21.15.64:0/2045225758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe812e0a0 con 0x7f0fe80b4e70 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fe77fe640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fe8145920 0x7f0fe81e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:14.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fe67fc640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe80b4e70 0x7f0fe812d870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c400 tx=0x7f0fd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:14.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fdffff640 1 -- 172.21.15.64:0/2045225758 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00074e0 con 0x7f0fe80b4e70 2024-03-07T07:19:14.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fdffff640 1 -- 172.21.15.64:0/2045225758 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0007680 con 0x7f0fe80b4e70 2024-03-07T07:19:14.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe812e380 con 0x7f0fe80b4e70 2024-03-07T07:19:14.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fdffff640 1 -- 172.21.15.64:0/2045225758 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0004a00 con 0x7f0fe80b4e70 2024-03-07T07:19:14.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.030+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe8071840 con 0x7f0fe80b4e70 2024-03-07T07:19:14.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.031+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe80754d0 con 0x7f0fe80b4e70 2024-03-07T07:19:14.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.031+0000 7f0fdffff640 1 -- 172.21.15.64:0/2045225758 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0fd001a020 con 0x7f0fe80b4e70 2024-03-07T07:19:14.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.032+0000 7f0fdffff640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0fc806b4d0 0x7f0fc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.032+0000 7f0fdffff640 1 -- 172.21.15.64:0/2045225758 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0fd00895c0 con 0x7f0fe80b4e70 2024-03-07T07:19:14.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.032+0000 7f0fe6ffd640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0fc806b4d0 0x7f0fc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.032+0000 7f0fe6ffd640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0fc806b4d0 0x7f0fc806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0036000 tx=0x7f0fe003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:14.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.036+0000 7f0fdffff640 1 -- 172.21.15.64:0/2045225758 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0fe80754d0 con 0x7f0fe80b4e70 2024-03-07T07:19:14.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.217+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0fe80b6650 con 0x7f0fe80b4e70 2024-03-07T07:19:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.271+0000 7f0fdffff640 1 -- 172.21.15.64:0/2045225758 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0fd0058820 con 0x7f0fe80b4e70 2024-03-07T07:19:14.272 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-03-07T07:19:14.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0fc806b4d0 msgr2=0x7f0fc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0fc806b4d0 0x7f0fc806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0036000 tx=0x7f0fe003c040 comp rx=0 tx=0).stop 2024-03-07T07:19:14.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe80b4e70 msgr2=0x7f0fe812d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe80b4e70 0x7f0fe812d870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c400 tx=0x7f0fd000c8c0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 shutdown_connections 2024-03-07T07:19:14.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0fc806b4d0 0x7f0fc806d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fe80b4e70 0x7f0fe812d870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fe8145920 0x7f0fe81e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 --2- 172.21.15.64:0/2045225758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fe806bc20 0x7f0fe8159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 >> 172.21.15.64:0/2045225758 conn(0x7f0fe8066e50 msgr2=0x7f0fe8142cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:14.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 shutdown_connections 2024-03-07T07:19:14.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.274+0000 7f0fed8a5640 1 -- 172.21.15.64:0/2045225758 wait complete. 2024-03-07T07:19:14.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-07T07:19:14.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.396+0000 7f6cc8f1d640 1 Processor -- start 2024-03-07T07:19:14.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.396+0000 7f6cc8f1d640 1 -- start start 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc8f1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6cc41413c0 0x7f6cc40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc8f1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc4072de0 0x7f6cc40aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc8f1d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc415eaa0 con 0x7f6cc40aaa90 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc8f1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc4073220 con 0x7f6cc41413c0 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc8f1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc415ed80 con 0x7f6cc4072de0 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6cc41413c0 0x7f6cc40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6cc41413c0 0x7f6cc40728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40634/0 (socket says 172.21.15.64:40634) 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc259c640 1 -- 172.21.15.64:0/3188080827 learned_addr learned my addr 172.21.15.64:0/3188080827 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc1d9b640 1 --2- 172.21.15.64:0/3188080827 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc4072de0 0x7f6cc40aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc259c640 1 -- 172.21.15.64:0/3188080827 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc4072de0 msgr2=0x7f6cc40aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc259c640 1 --2- 172.21.15.64:0/3188080827 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc4072de0 0x7f6cc40aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc259c640 1 -- 172.21.15.64:0/3188080827 >> v1:172.21.15.120:6789/0 conn(0x7f6cc40aaa90 legacy=0x7f6cc40a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.397+0000 7f6cc259c640 1 -- 172.21.15.64:0/3188080827 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc415f060 con 0x7f6cc41413c0 2024-03-07T07:19:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.398+0000 7f6cc259c640 1 --2- 172.21.15.64:0/3188080827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc40728a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009870 tx=0x7f6cb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdcd9e79159134c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.398+0000 7f6cc159a640 1 -- 172.21.15.64:0/3188080827 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4035020 con 0x7f6cc41413c0 2024-03-07T07:19:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.398+0000 7f6cc159a640 1 -- 172.21.15.64:0/3188080827 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4031640 con 0x7f6cc41413c0 2024-03-07T07:19:14.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.398+0000 7f6cc159a640 1 -- 172.21.15.64:0/3188080827 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb402d040 con 0x7f6cc41413c0 2024-03-07T07:19:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.398+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/3188080827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 msgr2=0x7f6cc40728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.398+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/3188080827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc40728a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009870 tx=0x7f6cb402b8f0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.399+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/3188080827 shutdown_connections 2024-03-07T07:19:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.399+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/3188080827 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc4072de0 0x7f6cc40aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.399+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/3188080827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc40728a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.399+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/3188080827 >> 172.21.15.64:0/3188080827 conn(0x7f6cc40cdf60 msgr2=0x7f6cc4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:14.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.399+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/3188080827 shutdown_connections 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.399+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/3188080827 wait complete. 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.400+0000 7f6cc8f1d640 1 Processor -- start 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.400+0000 7f6cc8f1d640 1 -- start start 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cc4072de0 0x7f6cc41f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc40aaa90 0x7f6cc4140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc4157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc40aaf20 con 0x7f6cc41413c0 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc415e760 con 0x7f6cc4072de0 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc40ab0a0 con 0x7f6cc40aaa90 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc4157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc4157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40646/0 (socket says 172.21.15.64:40646) 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 -- 172.21.15.64:0/959327541 learned_addr learned my addr 172.21.15.64:0/959327541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:14.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc259c640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cc4072de0 0x7f6cc41f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc1d9b640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc40aaa90 0x7f6cc4140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 -- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc40aaa90 msgr2=0x7f6cc4140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc40aaa90 0x7f6cc4140440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 -- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cc4072de0 msgr2=0x7f6cc41f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cc4072de0 0x7f6cc41f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 -- 172.21.15.64:0/959327541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40b1220 con 0x7f6cc41413c0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc2d9d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc4157b20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c7a0 tx=0x7f6cbc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cb2ffd640 1 -- 172.21.15.64:0/959327541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc003b70 con 0x7f6cc41413c0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cb2ffd640 1 -- 172.21.15.64:0/959327541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc003d30 con 0x7f6cc41413c0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc40b1500 con 0x7f6cc41413c0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cb2ffd640 1 -- 172.21.15.64:0/959327541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00aa90 con 0x7f6cc41413c0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.401+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc4140980 con 0x7f6cc41413c0 2024-03-07T07:19:14.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.402+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc4062c90 con 0x7f6cc41413c0 2024-03-07T07:19:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.403+0000 7f6cb2ffd640 1 -- 172.21.15.64:0/959327541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6cbc00ac30 con 0x7f6cc41413c0 2024-03-07T07:19:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.403+0000 7f6cb2ffd640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6ca406b500 0x7f6ca406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:14.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.404+0000 7f6cb2ffd640 1 -- 172.21.15.64:0/959327541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6cbc089d30 con 0x7f6cc41413c0 2024-03-07T07:19:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.404+0000 7f6cc259c640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6ca406b500 0x7f6ca406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:14.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.404+0000 7f6cc259c640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6ca406b500 0x7f6ca406d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4036000 tx=0x7f6cb403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:14.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.408+0000 7f6cb2ffd640 1 -- 172.21.15.64:0/959327541 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6cbc0587d0 con 0x7f6cc41413c0 2024-03-07T07:19:14.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:14.591+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6cc406ef10 con 0x7f6cc41413c0 2024-03-07T07:19:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.277+0000 7f6cb2ffd640 1 -- 172.21.15.64:0/959327541 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6cbc05c870 con 0x7f6cc41413c0 2024-03-07T07:19:15.277 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-03-07T07:19:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6ca406b500 msgr2=0x7f6ca406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6ca406b500 0x7f6ca406d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4036000 tx=0x7f6cb403c040 comp rx=0 tx=0).stop 2024-03-07T07:19:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 msgr2=0x7f6cc4157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc4157b20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00c7a0 tx=0x7f6cbc00cc60 comp rx=0 tx=0).stop 2024-03-07T07:19:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 shutdown_connections 2024-03-07T07:19:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6ca406b500 0x7f6ca406d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cc41413c0 0x7f6cc4157b20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cc40aaa90 0x7f6cc4140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 --2- 172.21.15.64:0/959327541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cc4072de0 0x7f6cc41f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.282+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 >> 172.21.15.64:0/959327541 conn(0x7f6cc40cdf60 msgr2=0x7f6cc40b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:15.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.283+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 shutdown_connections 2024-03-07T07:19:15.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.286+0000 7f6cc8f1d640 1 -- 172.21.15.64:0/959327541 wait complete. 2024-03-07T07:19:15.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-07T07:19:15.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.417+0000 7f0dc5027640 1 Processor -- start 2024-03-07T07:19:15.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.417+0000 7f0dc5027640 1 -- start start 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dc5027640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc015e350 0x7f0dc0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dc5027640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0dc00b7200 0x7f0dc01623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dc5027640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc0071ff0 con 0x7f0dc0071280 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dc5027640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc00d2160 con 0x7f0dc00b7200 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dc5027640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00d23b0 con 0x7f0dc015e350 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc015e350 0x7f0dc0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc015e350 0x7f0dc0161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:55714/0 (socket says 172.21.15.64:55714) 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dbed9d640 1 -- 172.21.15.64:0/1088340378 learned_addr learned my addr 172.21.15.64:0/1088340378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.418+0000 7f0dbe59c640 1 --2- 172.21.15.64:0/1088340378 >> v2:172.21.15.120:3300/0 conn(0x7f0dc00b7200 0x7f0dc01623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.419+0000 7f0dbed9d640 1 -- 172.21.15.64:0/1088340378 >> v2:172.21.15.120:3300/0 conn(0x7f0dc00b7200 msgr2=0x7f0dc01623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.419+0000 7f0dbed9d640 1 --2- 172.21.15.64:0/1088340378 >> v2:172.21.15.120:3300/0 conn(0x7f0dc00b7200 0x7f0dc01623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.419+0000 7f0dbed9d640 1 -- 172.21.15.64:0/1088340378 >> v1:172.21.15.120:6789/0 conn(0x7f0dc0071280 legacy=0x7f0dc0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:15.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.419+0000 7f0dbed9d640 1 -- 172.21.15.64:0/1088340378 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc00d2690 con 0x7f0dc015e350 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dbed9d640 1 --2- 172.21.15.64:0/1088340378 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc015e350 0x7f0dc0161eb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009870 tx=0x7f0db802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41fea469e1becd49 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dbdd9b640 1 -- 172.21.15.64:0/1088340378 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8034070 con 0x7f0dc015e350 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dbdd9b640 1 -- 172.21.15.64:0/1088340378 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db8030840 con 0x7f0dc015e350 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dbdd9b640 1 -- 172.21.15.64:0/1088340378 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8030c10 con 0x7f0dc015e350 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 -- 172.21.15.64:0/1088340378 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc015e350 msgr2=0x7f0dc0161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 --2- 172.21.15.64:0/1088340378 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc015e350 0x7f0dc0161eb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009870 tx=0x7f0db802b980 comp rx=0 tx=0).stop 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 -- 172.21.15.64:0/1088340378 shutdown_connections 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 --2- 172.21.15.64:0/1088340378 >> v2:172.21.15.120:3300/0 conn(0x7f0dc00b7200 0x7f0dc01623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 --2- 172.21.15.64:0/1088340378 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc015e350 0x7f0dc0161eb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 -- 172.21.15.64:0/1088340378 >> 172.21.15.64:0/1088340378 conn(0x7f0dc00d0e70 msgr2=0x7f0dc00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:15.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 -- 172.21.15.64:0/1088340378 shutdown_connections 2024-03-07T07:19:15.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.420+0000 7f0dc5027640 1 -- 172.21.15.64:0/1088340378 wait complete. 2024-03-07T07:19:15.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.422+0000 7f0dc5027640 1 Processor -- start 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.422+0000 7f0dc5027640 1 -- start start 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.422+0000 7f0dc5027640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc0071280 0x7f0dc00c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.422+0000 7f0dc5027640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc00b7200 0x7f0dc014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.422+0000 7f0dc5027640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc015e350 0x7f0dc00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.422+0000 7f0dc5027640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00d2e90 con 0x7f0dc015e350 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.422+0000 7f0dc5027640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc015e730 con 0x7f0dc00b7200 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dc5027640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00d3010 con 0x7f0dc0071280 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc015e350 0x7f0dc00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc015e350 0x7f0dc00c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40652/0 (socket says 172.21.15.64:40652) 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 -- 172.21.15.64:0/330036175 learned_addr learned my addr 172.21.15.64:0/330036175 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbe59c640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc00b7200 0x7f0dc014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:15.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbed9d640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc0071280 0x7f0dc00c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 -- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc0071280 msgr2=0x7f0dc00c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc0071280 0x7f0dc00c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 -- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc00b7200 msgr2=0x7f0dc014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc00b7200 0x7f0dc014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 -- 172.21.15.64:0/330036175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc014b7f0 con 0x7f0dc015e350 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dbf59e640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc015e350 0x7f0dc00c9e60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c7a0 tx=0x7f0db000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0db77fe640 1 -- 172.21.15.64:0/330036175 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0003b70 con 0x7f0dc015e350 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0db77fe640 1 -- 172.21.15.64:0/330036175 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0003d30 con 0x7f0dc015e350 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc00c80c0 con 0x7f0dc015e350 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0db77fe640 1 -- 172.21.15.64:0/330036175 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db000aa90 con 0x7f0dc015e350 2024-03-07T07:19:15.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.423+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc00c8680 con 0x7f0dc015e350 2024-03-07T07:19:15.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.424+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc0062c90 con 0x7f0dc015e350 2024-03-07T07:19:15.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.425+0000 7f0db77fe640 1 -- 172.21.15.64:0/330036175 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0db000ac30 con 0x7f0dc015e350 2024-03-07T07:19:15.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.425+0000 7f0db77fe640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:15.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.425+0000 7f0db77fe640 1 -- 172.21.15.64:0/330036175 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0db0089fa0 con 0x7f0dc015e350 2024-03-07T07:19:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.425+0000 7f0dbed9d640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:15.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.426+0000 7f0dbed9d640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0da006b4d0 0x7f0da006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0db8036000 tx=0x7f0db803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:15.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.430+0000 7f0db77fe640 1 -- 172.21.15.64:0/330036175 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0db0058830 con 0x7f0dc015e350 2024-03-07T07:19:15.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:15.651+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0dc015c4b0 con 0x7f0dc015e350 2024-03-07T07:19:15.725 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:19:15.727 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:19:16.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.289+0000 7f0db77fe640 1 -- 172.21.15.64:0/330036175 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0db005c8d0 con 0x7f0dc015e350 2024-03-07T07:19:16.289 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.292+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0da006b4d0 msgr2=0x7f0da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.292+0000 7f0dc5027640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0da006b4d0 0x7f0da006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0db8036000 tx=0x7f0db803b040 comp rx=0 tx=0).stop 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.292+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc015e350 msgr2=0x7f0dc00c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.292+0000 7f0dc5027640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc015e350 0x7f0dc00c9e60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c7a0 tx=0x7f0db000cc60 comp rx=0 tx=0).stop 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 shutdown_connections 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0da006b4d0 0x7f0da006d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dc015e350 0x7f0dc00c9e60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0dc00b7200 0x7f0dc014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 --2- 172.21.15.64:0/330036175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0dc0071280 0x7f0dc00c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 >> 172.21.15.64:0/330036175 conn(0x7f0dc00d0e70 msgr2=0x7f0dc00cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 shutdown_connections 2024-03-07T07:19:16.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.293+0000 7f0dc5027640 1 -- 172.21.15.64:0/330036175 wait complete. 2024-03-07T07:19:16.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.415+0000 7f1e3791d640 1 Processor -- start 2024-03-07T07:19:16.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.416+0000 7f1e3791d640 1 -- start start 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.416+0000 7f1e3791d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e300b7e50 0x7f1e300d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.416+0000 7f1e3791d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e300b73d0 con 0x7f1e3015dd50 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.416+0000 7f1e3791d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e300b4fa0 con 0x7f1e30159cb0 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.416+0000 7f1e3791d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e300b5280 con 0x7f1e300b7e50 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e34eb8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e300b7e50 0x7f1e300d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e34eb8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1e300b7e50 0x7f1e300d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40658/0 (socket says 172.21.15.64:40658) 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e35eba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1e3015dd50 0x7f1e3015c430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37400/0 (socket says 172.21.15.64:37400) 2024-03-07T07:19:16.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e34eb8640 1 -- 172.21.15.64:0/3955811675 learned_addr learned my addr 172.21.15.64:0/3955811675 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e34eb8640 1 -- 172.21.15.64:0/3955811675 >> v1:172.21.15.120:6789/0 conn(0x7f1e30159cb0 legacy=0x7f1e3015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e34eb8640 1 -- 172.21.15.64:0/3955811675 >> v1:172.21.15.64:6789/0 conn(0x7f1e3015dd50 legacy=0x7f1e3015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e34eb8640 1 -- 172.21.15.64:0/3955811675 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300b7650 con 0x7f1e300b7e50 2024-03-07T07:19:16.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.417+0000 7f1e27fff640 1 -- 172.21.15.64:0/3955811675 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119298038 0 0) 0x7f1e28001490 con 0x7f1e3015dd50 2024-03-07T07:19:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e34eb8640 1 --2- 172.21.15.64:0/3955811675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300d18f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009870 tx=0x7f1e2c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601d69bc559d6f2d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e27fff640 1 -- 172.21.15.64:0/3955811675 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c02e070 con 0x7f1e300b7e50 2024-03-07T07:19:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e27fff640 1 -- 172.21.15.64:0/3955811675 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c037070 con 0x7f1e300b7e50 2024-03-07T07:19:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e27fff640 1 -- 172.21.15.64:0/3955811675 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c031a00 con 0x7f1e300b7e50 2024-03-07T07:19:16.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e3791d640 1 -- 172.21.15.64:0/3955811675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 msgr2=0x7f1e300d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e3791d640 1 --2- 172.21.15.64:0/3955811675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300d18f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009870 tx=0x7f1e2c02b880 comp rx=0 tx=0).stop 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e3791d640 1 -- 172.21.15.64:0/3955811675 shutdown_connections 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e3791d640 1 --2- 172.21.15.64:0/3955811675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300d18f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e3791d640 1 -- 172.21.15.64:0/3955811675 >> 172.21.15.64:0/3955811675 conn(0x7f1e300d2090 msgr2=0x7f1e300d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e3791d640 1 -- 172.21.15.64:0/3955811675 shutdown_connections 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.418+0000 7f1e3791d640 1 -- 172.21.15.64:0/3955811675 wait complete. 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 Processor -- start 2024-03-07T07:19:16.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 -- start start 2024-03-07T07:19:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e3015dd50 0x7f1e30156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e30156f90 0x7f1e301f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3015cb40 con 0x7f1e300b7e50 2024-03-07T07:19:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3015ccc0 con 0x7f1e30156f90 2024-03-07T07:19:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e3791d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3015ce40 con 0x7f1e3015dd50 2024-03-07T07:19:16.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e356b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e356b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40668/0 (socket says 172.21.15.64:40668) 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.420+0000 7f1e356b9640 1 -- 172.21.15.64:0/2451489844 learned_addr learned my addr 172.21.15.64:0/2451489844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e356b9640 1 -- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e3015dd50 msgr2=0x7f1e30156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e356b9640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e3015dd50 0x7f1e30156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e356b9640 1 -- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e30156f90 msgr2=0x7f1e301f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e356b9640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e30156f90 0x7f1e301f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e356b9640 1 -- 172.21.15.64:0/2451489844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e301e58b0 con 0x7f1e300b7e50 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e356b9640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300c8230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000c460 tx=0x7f1e2000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e25ffb640 1 -- 172.21.15.64:0/2451489844 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20016020 con 0x7f1e300b7e50 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e25ffb640 1 -- 172.21.15.64:0/2451489844 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20012870 con 0x7f1e300b7e50 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e301f2cb0 con 0x7f1e300b7e50 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e25ffb640 1 -- 172.21.15.64:0/2451489844 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20012c60 con 0x7f1e300b7e50 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.421+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e301e5e50 con 0x7f1e300b7e50 2024-03-07T07:19:16.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.422+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e30062c90 con 0x7f1e300b7e50 2024-03-07T07:19:16.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.422+0000 7f1e25ffb640 1 -- 172.21.15.64:0/2451489844 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1e20019020 con 0x7f1e300b7e50 2024-03-07T07:19:16.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.423+0000 7f1e25ffb640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e1406b4d0 0x7f1e1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.423+0000 7f1e25ffb640 1 -- 172.21.15.64:0/2451489844 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e20089bf0 con 0x7f1e300b7e50 2024-03-07T07:19:16.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.423+0000 7f1e34eb8640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e1406b4d0 0x7f1e1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.423+0000 7f1e34eb8640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e1406b4d0 0x7f1e1406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c036000 tx=0x7f1e2c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:16.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.427+0000 7f1e25ffb640 1 -- 172.21.15.64:0/2451489844 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e20020070 con 0x7f1e300b7e50 2024-03-07T07:19:16.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.608+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e3006dc70 con 0x7f1e300b7e50 2024-03-07T07:19:16.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.609+0000 7f1e25ffb640 1 -- 172.21.15.64:0/2451489844 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f1e20058b00 con 0x7f1e300b7e50 2024-03-07T07:19:16.609 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:16.609 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:15.653034+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:16.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e1406b4d0 msgr2=0x7f1e1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e1406b4d0 0x7f1e1406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c036000 tx=0x7f1e2c030000 comp rx=0 tx=0).stop 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 msgr2=0x7f1e300c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300c8230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000c460 tx=0x7f1e2000c920 comp rx=0 tx=0).stop 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 shutdown_connections 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e1406b4d0 0x7f1e1406d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e30156f90 0x7f1e301f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e3015dd50 0x7f1e30156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 --2- 172.21.15.64:0/2451489844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e300b7e50 0x7f1e300c8230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 >> 172.21.15.64:0/2451489844 conn(0x7f1e300d2090 msgr2=0x7f1e3015d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 shutdown_connections 2024-03-07T07:19:16.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.611+0000 7f1e3791d640 1 -- 172.21.15.64:0/2451489844 wait complete. 2024-03-07T07:19:16.622 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:16.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.723+0000 7fa4574dd640 1 Processor -- start 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.723+0000 7fa4574dd640 1 -- start start 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa4574dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa4500b6020 0x7fa4500b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa4574dd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa45015f880 0x7fa45015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa4574dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45015e660 con 0x7fa450147b50 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa4574dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45006e6c0 con 0x7fa4500b6020 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa4574dd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500b3160 con 0x7fa45015f880 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa454a78640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa4500b6020 0x7fa4500b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa454a78640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa4500b6020 0x7fa4500b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40676/0 (socket says 172.21.15.64:40676) 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa454a78640 1 -- 172.21.15.64:0/1112033401 learned_addr learned my addr 172.21.15.64:0/1112033401 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.724+0000 7fa455a7a640 1 --2- 172.21.15.64:0/1112033401 >> v2:172.21.15.120:3300/0 conn(0x7fa45015f880 0x7fa45015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.725+0000 7fa454a78640 1 -- 172.21.15.64:0/1112033401 >> v2:172.21.15.120:3300/0 conn(0x7fa45015f880 msgr2=0x7fa45015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.725+0000 7fa454a78640 1 --2- 172.21.15.64:0/1112033401 >> v2:172.21.15.120:3300/0 conn(0x7fa45015f880 0x7fa45015fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.725+0000 7fa454a78640 1 -- 172.21.15.64:0/1112033401 >> v1:172.21.15.120:6789/0 conn(0x7fa450147b50 legacy=0x7fa45015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.725+0000 7fa454a78640 1 -- 172.21.15.64:0/1112033401 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500b3440 con 0x7fa4500b6020 2024-03-07T07:19:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.725+0000 7fa454a78640 1 --2- 172.21.15.64:0/1112033401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4500b6020 0x7fa4500b2ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa4500b0a60 tx=0x7fa44402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf8d34dfe1172b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa44ffff640 1 -- 172.21.15.64:0/1112033401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444034070 con 0x7fa4500b6020 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa44ffff640 1 -- 172.21.15.64:0/1112033401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444030860 con 0x7fa4500b6020 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa44ffff640 1 -- 172.21.15.64:0/1112033401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444030c30 con 0x7fa4500b6020 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 -- 172.21.15.64:0/1112033401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4500b6020 msgr2=0x7fa4500b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 --2- 172.21.15.64:0/1112033401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4500b6020 0x7fa4500b2ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa4500b0a60 tx=0x7fa44402b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 -- 172.21.15.64:0/1112033401 shutdown_connections 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 --2- 172.21.15.64:0/1112033401 >> v2:172.21.15.120:3300/0 conn(0x7fa45015f880 0x7fa45015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 --2- 172.21.15.64:0/1112033401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4500b6020 0x7fa4500b2ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 -- 172.21.15.64:0/1112033401 >> 172.21.15.64:0/1112033401 conn(0x7fa45006b040 msgr2=0x7fa45006b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:16.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 -- 172.21.15.64:0/1112033401 shutdown_connections 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.726+0000 7fa4574dd640 1 -- 172.21.15.64:0/1112033401 wait complete. 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.727+0000 7fa4574dd640 1 Processor -- start 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.727+0000 7fa4574dd640 1 -- start start 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa4574dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4500b6020 0x7fa4501e9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa4574dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa450147b50 0x7fa45015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa4574dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa45015f880 0x7fa45015bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa4574dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45015f500 con 0x7fa450147b50 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa4574dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45015e8d0 con 0x7fa4500b6020 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa4574dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45015f680 con 0x7fa45015f880 2024-03-07T07:19:16.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa450147b50 0x7fa45015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa450147b50 0x7fa45015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40682/0 (socket says 172.21.15.64:40682) 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 -- 172.21.15.64:0/3743134747 learned_addr learned my addr 172.21.15.64:0/3743134747 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa455279640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4500b6020 0x7fa4501e9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa455a7a640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa45015f880 0x7fa45015bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 -- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa45015f880 msgr2=0x7fa45015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa45015f880 0x7fa45015bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 -- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4500b6020 msgr2=0x7fa4501e9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4500b6020 0x7fa4501e9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.728+0000 7fa454a78640 1 -- 172.21.15.64:0/3743134747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500afa80 con 0x7fa450147b50 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.729+0000 7fa454a78640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa450147b50 0x7fa45015b690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa44402b9e0 tx=0x7fa444004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.729+0000 7fa44dffb640 1 -- 172.21.15.64:0/3743134747 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444048070 con 0x7fa450147b50 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.729+0000 7fa44dffb640 1 -- 172.21.15.64:0/3743134747 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444030660 con 0x7fa450147b50 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.729+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4501e69a0 con 0x7fa450147b50 2024-03-07T07:19:16.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.729+0000 7fa44dffb640 1 -- 172.21.15.64:0/3743134747 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444034040 con 0x7fa450147b50 2024-03-07T07:19:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.729+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4501e6f60 con 0x7fa450147b50 2024-03-07T07:19:16.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.730+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450062c90 con 0x7fa450147b50 2024-03-07T07:19:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.730+0000 7fa44dffb640 1 -- 172.21.15.64:0/3743134747 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa44404b070 con 0x7fa450147b50 2024-03-07T07:19:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.731+0000 7fa44dffb640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.731+0000 7fa44dffb640 1 -- 172.21.15.64:0/3743134747 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4440ac0b0 con 0x7fa450147b50 2024-03-07T07:19:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.731+0000 7fa455279640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:16.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.731+0000 7fa455279640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa43c06b4d0 0x7fa43c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa448001f10 tx=0x7fa448009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:16.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.735+0000 7fa44dffb640 1 -- 172.21.15.64:0/3743134747 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa44407a8e0 con 0x7fa450147b50 2024-03-07T07:19:16.755 INFO:tasks.ceph.mds.b.smithi120.stderr:2024-03-07T07:19:16.755+0000 7fd3e39f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T07:19:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.916+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4500afd60 con 0x7fa450147b50 2024-03-07T07:19:16.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.917+0000 7fa44dffb640 1 -- 172.21.15.64:0/3743134747 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16582 (secure 0 0 0) 0x7fa44407e980 con 0x7fa450147b50 2024-03-07T07:19:16.917 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:16.917 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":19,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:15.653034+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:16.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa43c06b4d0 msgr2=0x7fa43c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa43c06b4d0 0x7fa43c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa448001f10 tx=0x7fa448009040 comp rx=0 tx=0).stop 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa450147b50 msgr2=0x7fa45015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa450147b50 0x7fa45015b690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa44402b9e0 tx=0x7fa444004730 comp rx=0 tx=0).stop 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 shutdown_connections 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa450147b50 0x7fa45015b690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa45015f880 0x7fa45015bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 --2- 172.21.15.64:0/3743134747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4500b6020 0x7fa4501e9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 >> 172.21.15.64:0/3743134747 conn(0x7fa45006b040 msgr2=0x7fa450160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 shutdown_connections 2024-03-07T07:19:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:16.919+0000 7fa4574dd640 1 -- 172.21.15.64:0/3743134747 wait complete. 2024-03-07T07:19:16.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-07T07:19:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.034+0000 7fb9282dd640 1 Processor -- start 2024-03-07T07:19:17.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.034+0000 7fb9282dd640 1 -- start start 2024-03-07T07:19:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb9282dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9200cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb9282dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9200d3310 0x7fb920167e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb9282dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb920070eb0 con 0x7fb9200cea70 2024-03-07T07:19:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb9282dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb920071190 con 0x7fb9200d3310 2024-03-07T07:19:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb9282dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb920071470 con 0x7fb92015e870 2024-03-07T07:19:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb92687a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9200d3310 0x7fb920167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb92687a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb9200d3310 0x7fb920167e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40688/0 (socket says 172.21.15.64:40688) 2024-03-07T07:19:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb92687a640 1 -- 172.21.15.64:0/133077607 learned_addr learned my addr 172.21.15.64:0/133077607 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb926079640 1 --2- 172.21.15.64:0/133077607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9200cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb92687a640 1 -- 172.21.15.64:0/133077607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 msgr2=0x7fb9200cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb92687a640 1 --2- 172.21.15.64:0/133077607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9200cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb92687a640 1 -- 172.21.15.64:0/133077607 >> v1:172.21.15.120:6789/0 conn(0x7fb9200cea70 legacy=0x7fb92015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb925077640 1 -- 172.21.15.64:0/133077607 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402568229 0 0) 0x7fb920070eb0 con 0x7fb9200cea70 2024-03-07T07:19:17.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.035+0000 7fb92687a640 1 -- 172.21.15.64:0/133077607 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb920071750 con 0x7fb9200d3310 2024-03-07T07:19:17.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.036+0000 7fb92687a640 1 --2- 172.21.15.64:0/133077607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200d3310 0x7fb920167e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb918009e50 tx=0x7fb91802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168c0a762dd60289 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.036+0000 7fb925077640 1 -- 172.21.15.64:0/133077607 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91802cb00 con 0x7fb9200d3310 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.036+0000 7fb925077640 1 -- 172.21.15.64:0/133077607 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91802cca0 con 0x7fb9200d3310 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.036+0000 7fb925077640 1 -- 172.21.15.64:0/133077607 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918034610 con 0x7fb9200d3310 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.036+0000 7fb9282dd640 1 -- 172.21.15.64:0/133077607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200d3310 msgr2=0x7fb920167e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.036+0000 7fb9282dd640 1 --2- 172.21.15.64:0/133077607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200d3310 0x7fb920167e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb918009e50 tx=0x7fb91802bf60 comp rx=0 tx=0).stop 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.037+0000 7fb9282dd640 1 -- 172.21.15.64:0/133077607 shutdown_connections 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.037+0000 7fb9282dd640 1 --2- 172.21.15.64:0/133077607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200d3310 0x7fb920167e20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.037+0000 7fb9282dd640 1 --2- 172.21.15.64:0/133077607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9200cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.037+0000 7fb9282dd640 1 -- 172.21.15.64:0/133077607 >> 172.21.15.64:0/133077607 conn(0x7fb9200d11e0 msgr2=0x7fb9200d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.037+0000 7fb9282dd640 1 -- 172.21.15.64:0/133077607 shutdown_connections 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.037+0000 7fb9282dd640 1 -- 172.21.15.64:0/133077607 wait complete. 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 Processor -- start 2024-03-07T07:19:17.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 -- start start 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200cea70 0x7fb9200c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9200d3310 0x7fb92015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9201603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200cf6d0 con 0x7fb9200cea70 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb920070b60 con 0x7fb9200d3310 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb9282dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200cf850 con 0x7fb92015e870 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.038+0000 7fb926079640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200cea70 0x7fb9200c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200cea70 0x7fb9200c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40694/0 (socket says 172.21.15.64:40694) 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 -- 172.21.15.64:0/1150416660 learned_addr learned my addr 172.21.15.64:0/1150416660 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb92687a640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9201603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb925878640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9200d3310 0x7fb92015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 -- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 msgr2=0x7fb9201603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9201603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 -- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9200d3310 msgr2=0x7fb92015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9200d3310 0x7fb92015feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 -- 172.21.15.64:0/1150416660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9201f7f50 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb926079640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200cea70 0x7fb9200c9b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00e9b0 tx=0x7fb91c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb916ffd640 1 -- 172.21.15.64:0/1150416660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c017070 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb916ffd640 1 -- 172.21.15.64:0/1150416660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c00c9a0 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9201f8230 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb916ffd640 1 -- 172.21.15.64:0/1150416660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00cd90 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.039+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9200ca080 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.040+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb920062c90 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.040+0000 7fb916ffd640 1 -- 172.21.15.64:0/1150416660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb91c01a070 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.041+0000 7fb916ffd640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb90406b4d0 0x7fb90406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.041+0000 7fb916ffd640 1 -- 172.21.15.64:0/1150416660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb91c08a4a0 con 0x7fb9200cea70 2024-03-07T07:19:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.041+0000 7fb925878640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb90406b4d0 0x7fb90406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.041+0000 7fb925878640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb90406b4d0 0x7fb90406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb920151ec0 tx=0x7fb910009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.045+0000 7fb916ffd640 1 -- 172.21.15.64:0/1150416660 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb91c058d60 con 0x7fb9200cea70 2024-03-07T07:19:17.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.266+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb920153f90 con 0x7fb9200cea70 2024-03-07T07:19:17.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.658+0000 7fb916ffd640 1 -- 172.21.15.64:0/1150416660 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb91c05ce00 con 0x7fb9200cea70 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb90406b4d0 msgr2=0x7fb90406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb90406b4d0 0x7fb90406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb920151ec0 tx=0x7fb910009040 comp rx=0 tx=0).stop 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200cea70 msgr2=0x7fb9200c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200cea70 0x7fb9200c9b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00e9b0 tx=0x7fb91c00ee70 comp rx=0 tx=0).stop 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 shutdown_connections 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb90406b4d0 0x7fb90406d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb92015e870 0x7fb9201603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb9200d3310 0x7fb92015feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 --2- 172.21.15.64:0/1150416660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9200cea70 0x7fb9200c9b40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.660+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 >> 172.21.15.64:0/1150416660 conn(0x7fb9200d11e0 msgr2=0x7fb92015d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.661+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 shutdown_connections 2024-03-07T07:19:17.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.661+0000 7fb9282dd640 1 -- 172.21.15.64:0/1150416660 wait complete. 2024-03-07T07:19:17.672 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-07T07:19:17.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-07T07:19:17.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.777+0000 7f53e9b99640 1 Processor -- start 2024-03-07T07:19:17.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.777+0000 7f53e9b99640 1 -- start start 2024-03-07T07:19:17.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e9b99640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53e415e350 0x7f53e4161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e9b99640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53e4071990 0x7f53e41623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e9b99640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e4071ff0 con 0x7f53e40b7200 2024-03-07T07:19:17.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e9b99640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40d2160 con 0x7f53e415e350 2024-03-07T07:19:17.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e9b99640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40d23b0 con 0x7f53e4071990 2024-03-07T07:19:17.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53e415e350 0x7f53e4161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f53e415e350 0x7f53e4161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40706/0 (socket says 172.21.15.64:40706) 2024-03-07T07:19:17.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e2ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f53e40b7200 0x7f53e4071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37410/0 (socket says 172.21.15.64:37410) 2024-03-07T07:19:17.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e37fe640 1 -- 172.21.15.64:0/2215055973 learned_addr learned my addr 172.21.15.64:0/2215055973 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:17.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e3fff640 1 --2- 172.21.15.64:0/2215055973 >> v2:172.21.15.120:3300/0 conn(0x7f53e4071990 0x7f53e41623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e37fe640 1 -- 172.21.15.64:0/2215055973 >> v2:172.21.15.120:3300/0 conn(0x7f53e4071990 msgr2=0x7f53e41623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:17.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e37fe640 1 --2- 172.21.15.64:0/2215055973 >> v2:172.21.15.120:3300/0 conn(0x7f53e4071990 0x7f53e41623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e37fe640 1 -- 172.21.15.64:0/2215055973 >> v1:172.21.15.64:6789/0 conn(0x7f53e40b7200 legacy=0x7f53e4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e37fe640 1 -- 172.21.15.64:0/2215055973 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40d2690 con 0x7f53e415e350 2024-03-07T07:19:17.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.778+0000 7f53e27fc640 1 -- 172.21.15.64:0/2215055973 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500668174 0 0) 0x7f53e4071ff0 con 0x7f53e40b7200 2024-03-07T07:19:17.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.779+0000 7f53e37fe640 1 --2- 172.21.15.64:0/2215055973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e415e350 0x7f53e4161eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53e408c7f0 tx=0x7f53d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9cedf01f215392 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:17.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.779+0000 7f53e27fc640 1 -- 172.21.15.64:0/2215055973 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8034070 con 0x7f53e415e350 2024-03-07T07:19:17.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.779+0000 7f53e27fc640 1 -- 172.21.15.64:0/2215055973 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d8030860 con 0x7f53e415e350 2024-03-07T07:19:17.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.779+0000 7f53e27fc640 1 -- 172.21.15.64:0/2215055973 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8030c30 con 0x7f53e415e350 2024-03-07T07:19:17.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.779+0000 7f53e9b99640 1 -- 172.21.15.64:0/2215055973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e415e350 msgr2=0x7f53e4161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.779+0000 7f53e9b99640 1 --2- 172.21.15.64:0/2215055973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e415e350 0x7f53e4161eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53e408c7f0 tx=0x7f53d802b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.780+0000 7f53e9b99640 1 -- 172.21.15.64:0/2215055973 shutdown_connections 2024-03-07T07:19:17.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.780+0000 7f53e9b99640 1 --2- 172.21.15.64:0/2215055973 >> v2:172.21.15.120:3300/0 conn(0x7f53e4071990 0x7f53e41623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.780+0000 7f53e9b99640 1 --2- 172.21.15.64:0/2215055973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e415e350 0x7f53e4161eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.780+0000 7f53e9b99640 1 -- 172.21.15.64:0/2215055973 >> 172.21.15.64:0/2215055973 conn(0x7f53e40d0e70 msgr2=0x7f53e40d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:17.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.780+0000 7f53e9b99640 1 -- 172.21.15.64:0/2215055973 shutdown_connections 2024-03-07T07:19:17.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.780+0000 7f53e9b99640 1 -- 172.21.15.64:0/2215055973 wait complete. 2024-03-07T07:19:17.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.781+0000 7f53e9b99640 1 Processor -- start 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.781+0000 7f53e9b99640 1 -- start start 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53e4071990 0x7f53e41f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e40b7200 0x7f53e41f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53e415e350 0x7f53e415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40d2e90 con 0x7f53e40b7200 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e415e730 con 0x7f53e415e350 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40d3010 con 0x7f53e4071990 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e40b7200 0x7f53e41f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e40b7200 0x7f53e41f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40722/0 (socket says 172.21.15.64:40722) 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 -- 172.21.15.64:0/1246020075 learned_addr learned my addr 172.21.15.64:0/1246020075 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:17.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e3fff640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53e415e350 0x7f53e415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e37fe640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53e4071990 0x7f53e41f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 -- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53e4071990 msgr2=0x7f53e41f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53e4071990 0x7f53e41f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 -- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53e415e350 msgr2=0x7f53e415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53e415e350 0x7f53e415e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 -- 172.21.15.64:0/1246020075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e415f020 con 0x7f53e40b7200 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e2ffd640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e40b7200 0x7f53e41f14f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c4d0 tx=0x7f53d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53cbfff640 1 -- 172.21.15.64:0/1246020075 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4018070 con 0x7f53e40b7200 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53cbfff640 1 -- 172.21.15.64:0/1246020075 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d4007500 con 0x7f53e40b7200 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e415f300 con 0x7f53e40b7200 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53cbfff640 1 -- 172.21.15.64:0/1246020075 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4004a20 con 0x7f53e40b7200 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.782+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e415f8c0 con 0x7f53e40b7200 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.783+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e4062c90 con 0x7f53e40b7200 2024-03-07T07:19:17.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.784+0000 7f53cbfff640 1 -- 172.21.15.64:0/1246020075 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f53d401b070 con 0x7f53e40b7200 2024-03-07T07:19:17.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.784+0000 7f53cbfff640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:17.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.784+0000 7f53cbfff640 1 -- 172.21.15.64:0/1246020075 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53d40893d0 con 0x7f53e40b7200 2024-03-07T07:19:17.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.784+0000 7f53e37fe640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:17.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.785+0000 7f53e37fe640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f53d8036000 tx=0x7f53d803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:17.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:17.789+0000 7f53cbfff640 1 -- 172.21.15.64:0/1246020075 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f53d4057c90 con 0x7f53e40b7200 2024-03-07T07:19:18.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.010+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f53e40d1d10 con 0x7f53e40b7200 2024-03-07T07:19:18.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.661+0000 7f53cbfff640 1 -- 172.21.15.64:0/1246020075 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53d405bd30 con 0x7f53e40b7200 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.663+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f53d006b4d0 msgr2=0x7f53d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.663+0000 7f53e9b99640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f53d8036000 tx=0x7f53d803c040 comp rx=0 tx=0).stop 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.663+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e40b7200 msgr2=0x7f53e41f14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.663+0000 7f53e9b99640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e40b7200 0x7f53e41f14f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c4d0 tx=0x7f53d400c990 comp rx=0 tx=0).stop 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 shutdown_connections 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53e40b7200 0x7f53e41f14f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53e415e350 0x7f53e415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 --2- 172.21.15.64:0/1246020075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53e4071990 0x7f53e41f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 >> 172.21.15.64:0/1246020075 conn(0x7f53e40d0e70 msgr2=0x7f53e415d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 shutdown_connections 2024-03-07T07:19:18.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.664+0000 7f53e9b99640 1 -- 172.21.15.64:0/1246020075 wait complete. 2024-03-07T07:19:18.678 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-07T07:19:18.678 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.784+0000 7f07212d0640 1 Processor -- start 2024-03-07T07:19:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.784+0000 7f07212d0640 1 -- start start 2024-03-07T07:19:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f07212d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f071c144540 0x7f071c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f07212d0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f071c162ab0 0x7f071c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:18.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f07212d0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c0b4bd0 con 0x7f071c0b5dd0 2024-03-07T07:19:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f07212d0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c141700 con 0x7f071c144540 2024-03-07T07:19:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f07212d0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c161e00 con 0x7f071c162ab0 2024-03-07T07:19:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f071c144540 0x7f071c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071affd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f071c0b5dd0 0x7f071c0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37424/0 (socket says 172.21.15.64:37424) 2024-03-07T07:19:18.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f071c144540 0x7f071c162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40736/0 (socket says 172.21.15.64:40736) 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071affd640 1 -- 172.21.15.64:0/1038556 learned_addr learned my addr 172.21.15.64:0/1038556 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071b7fe640 1 --2- 172.21.15.64:0/1038556 >> v2:172.21.15.120:3300/0 conn(0x7f071c162ab0 0x7f071c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071a7fc640 1 -- 172.21.15.64:0/1038556 >> v2:172.21.15.120:3300/0 conn(0x7f071c162ab0 msgr2=0x7f071c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071a7fc640 1 --2- 172.21.15.64:0/1038556 >> v2:172.21.15.120:3300/0 conn(0x7f071c162ab0 0x7f071c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071a7fc640 1 -- 172.21.15.64:0/1038556 >> v1:172.21.15.64:6789/0 conn(0x7f071c0b5dd0 legacy=0x7f071c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f0719ffb640 1 -- 172.21.15.64:0/1038556 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412996348 0 0) 0x7f071c0b4bd0 con 0x7f071c0b5dd0 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.785+0000 7f071a7fc640 1 -- 172.21.15.64:0/1038556 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0b4eb0 con 0x7f071c144540 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.786+0000 7f071a7fc640 1 --2- 172.21.15.64:0/1038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c144540 0x7f071c162570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f071c0722a0 tx=0x7f070402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f97f83e462d735 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.786+0000 7f0719ffb640 1 -- 172.21.15.64:0/1038556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07040315a0 con 0x7f071c144540 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.786+0000 7f0719ffb640 1 -- 172.21.15.64:0/1038556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0704037070 con 0x7f071c144540 2024-03-07T07:19:18.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.786+0000 7f0719ffb640 1 -- 172.21.15.64:0/1038556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704031960 con 0x7f071c144540 2024-03-07T07:19:18.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 -- 172.21.15.64:0/1038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c144540 msgr2=0x7f071c162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 --2- 172.21.15.64:0/1038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c144540 0x7f071c162570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f071c0722a0 tx=0x7f070402b840 comp rx=0 tx=0).stop 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 -- 172.21.15.64:0/1038556 shutdown_connections 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 --2- 172.21.15.64:0/1038556 >> v2:172.21.15.120:3300/0 conn(0x7f071c162ab0 0x7f071c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 --2- 172.21.15.64:0/1038556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c144540 0x7f071c162570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 -- 172.21.15.64:0/1038556 >> 172.21.15.64:0/1038556 conn(0x7f071c0d0e70 msgr2=0x7f071c0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 -- 172.21.15.64:0/1038556 shutdown_connections 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.787+0000 7f07212d0640 1 -- 172.21.15.64:0/1038556 wait complete. 2024-03-07T07:19:18.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.788+0000 7f07212d0640 1 Processor -- start 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.788+0000 7f07212d0640 1 -- start start 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f07212d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c0b5dd0 0x7f071c13edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f07212d0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f071c144540 0x7f071c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f07212d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f071c1603c0 0x7f071c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f07212d0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0b61b0 con 0x7f071c0b5dd0 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f07212d0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c162050 con 0x7f071c1603c0 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f07212d0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c071950 con 0x7f071c144540 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c0b5dd0 0x7f071c13edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:18.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c0b5dd0 0x7f071c13edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40746/0 (socket says 172.21.15.64:40746) 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 -- 172.21.15.64:0/3474904274 learned_addr learned my addr 172.21.15.64:0/3474904274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071b7fe640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f071c1603c0 0x7f071c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071a7fc640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f071c144540 0x7f071c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 -- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f071c144540 msgr2=0x7f071c15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f071c144540 0x7f071c15b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 -- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f071c1603c0 msgr2=0x7f071c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f071c1603c0 0x7f071c159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 -- 172.21.15.64:0/3474904274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c160850 con 0x7f071c0b5dd0 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.789+0000 7f071affd640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c0b5dd0 0x7f071c13edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f071400c4d0 tx=0x7f071400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.790+0000 7f07137fe640 1 -- 172.21.15.64:0/3474904274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714018070 con 0x7f071c0b5dd0 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.790+0000 7f07137fe640 1 -- 172.21.15.64:0/3474904274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0714007500 con 0x7f071c0b5dd0 2024-03-07T07:19:18.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.790+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c160b30 con 0x7f071c0b5dd0 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.790+0000 7f07137fe640 1 -- 172.21.15.64:0/3474904274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714004a20 con 0x7f071c0b5dd0 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.790+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c1610f0 con 0x7f071c0b5dd0 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.791+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c062c90 con 0x7f071c0b5dd0 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.791+0000 7f07137fe640 1 -- 172.21.15.64:0/3474904274 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f071401b070 con 0x7f071c0b5dd0 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.791+0000 7f07137fe640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.791+0000 7f07137fe640 1 -- 172.21.15.64:0/3474904274 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0714089490 con 0x7f071c0b5dd0 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.792+0000 7f071a7fc640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:18.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.792+0000 7f071a7fc640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f06fc06b4d0 0x7f06fc06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0704036000 tx=0x7f070403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:18.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:18.796+0000 7f07137fe640 1 -- 172.21.15.64:0/3474904274 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0714057cc0 con 0x7f071c0b5dd0 2024-03-07T07:19:19.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.014+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f071c1438a0 con 0x7f071c0b5dd0 2024-03-07T07:19:19.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.014+0000 7f07137fe640 1 -- 172.21.15.64:0/3474904274 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7f071405bd60 con 0x7f071c0b5dd0 2024-03-07T07:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:19.016 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:19.016 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:19.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f06fc06b4d0 msgr2=0x7f06fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f06fc06b4d0 0x7f06fc06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0704036000 tx=0x7f070403d040 comp rx=0 tx=0).stop 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c0b5dd0 msgr2=0x7f071c13edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c0b5dd0 0x7f071c13edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f071400c4d0 tx=0x7f071400c990 comp rx=0 tx=0).stop 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 shutdown_connections 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f071c1603c0 0x7f071c159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f071c144540 0x7f071c15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 --2- 172.21.15.64:0/3474904274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f071c0b5dd0 0x7f071c13edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 >> 172.21.15.64:0/3474904274 conn(0x7f071c0d0e70 msgr2=0x7f071c0b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 shutdown_connections 2024-03-07T07:19:19.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.016+0000 7f07212d0640 1 -- 172.21.15.64:0/3474904274 wait complete. 2024-03-07T07:19:19.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:19.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.128+0000 7f3603310640 1 Processor -- start 2024-03-07T07:19:19.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.128+0000 7f3603310640 1 -- start start 2024-03-07T07:19:19.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f3603310640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f35fc147b50 0x7f35fc0b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f3603310640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35fc0b2ad0 0x7f35fc15edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f3603310640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc15e660 con 0x7f35fc15f330 2024-03-07T07:19:19.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f3603310640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc06e6c0 con 0x7f35fc0b2ad0 2024-03-07T07:19:19.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f3603310640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0b2f40 con 0x7f35fc147b50 2024-03-07T07:19:19.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f36008ab640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35fc0b2ad0 0x7f35fc15edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f36008ab640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f35fc0b2ad0 0x7f35fc15edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40748/0 (socket says 172.21.15.64:40748) 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f36008ab640 1 -- 172.21.15.64:0/4202690736 learned_addr learned my addr 172.21.15.64:0/4202690736 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.129+0000 7f36010ac640 1 --2- 172.21.15.64:0/4202690736 >> v2:172.21.15.120:3300/0 conn(0x7f35fc147b50 0x7f35fc0b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.130+0000 7f36008ab640 1 -- 172.21.15.64:0/4202690736 >> v2:172.21.15.120:3300/0 conn(0x7f35fc147b50 msgr2=0x7f35fc0b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.130+0000 7f36008ab640 1 --2- 172.21.15.64:0/4202690736 >> v2:172.21.15.120:3300/0 conn(0x7f35fc147b50 0x7f35fc0b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.130+0000 7f36008ab640 1 -- 172.21.15.64:0/4202690736 >> v1:172.21.15.120:6789/0 conn(0x7f35fc15f330 legacy=0x7f35fc15f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.130+0000 7f36008ab640 1 -- 172.21.15.64:0/4202690736 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0b3160 con 0x7f35fc0b2ad0 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.130+0000 7f36008ab640 1 --2- 172.21.15.64:0/4202690736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc0b2ad0 0x7f35fc15edf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b0980 tx=0x7f35f002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd20800eccef8086 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f35fbfff640 1 -- 172.21.15.64:0/4202690736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0034070 con 0x7f35fc0b2ad0 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f35fbfff640 1 -- 172.21.15.64:0/4202690736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0030860 con 0x7f35fc0b2ad0 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f35fbfff640 1 -- 172.21.15.64:0/4202690736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0030c30 con 0x7f35fc0b2ad0 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 -- 172.21.15.64:0/4202690736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc0b2ad0 msgr2=0x7f35fc15edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 --2- 172.21.15.64:0/4202690736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc0b2ad0 0x7f35fc15edf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b0980 tx=0x7f35f002b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 -- 172.21.15.64:0/4202690736 shutdown_connections 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 --2- 172.21.15.64:0/4202690736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc0b2ad0 0x7f35fc15edf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 --2- 172.21.15.64:0/4202690736 >> v2:172.21.15.120:3300/0 conn(0x7f35fc147b50 0x7f35fc0b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 -- 172.21.15.64:0/4202690736 >> 172.21.15.64:0/4202690736 conn(0x7f35fc06b040 msgr2=0x7f35fc06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:19.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 -- 172.21.15.64:0/4202690736 shutdown_connections 2024-03-07T07:19:19.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.131+0000 7f3603310640 1 -- 172.21.15.64:0/4202690736 wait complete. 2024-03-07T07:19:19.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.132+0000 7f3603310640 1 Processor -- start 2024-03-07T07:19:19.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.132+0000 7f3603310640 1 -- start start 2024-03-07T07:19:19.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f3603310640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35fc0b2ad0 0x7f35fc1428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f3603310640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc147b50 0x7f35fc1e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f3603310640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f35fc15f330 0x7f35fc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f3603310640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc15e8d0 con 0x7f35fc147b50 2024-03-07T07:19:19.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f3603310640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0b3720 con 0x7f35fc0b2ad0 2024-03-07T07:19:19.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f3603310640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc15ff90 con 0x7f35fc15f330 2024-03-07T07:19:19.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc147b50 0x7f35fc1e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc147b50 0x7f35fc1e6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40752/0 (socket says 172.21.15.64:40752) 2024-03-07T07:19:19.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36010ac640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35fc0b2ad0 0x7f35fc1428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 -- 172.21.15.64:0/847279716 learned_addr learned my addr 172.21.15.64:0/847279716 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36018ad640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f35fc15f330 0x7f35fc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 -- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f35fc15f330 msgr2=0x7f35fc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f35fc15f330 0x7f35fc15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 -- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35fc0b2ad0 msgr2=0x7f35fc1428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35fc0b2ad0 0x7f35fc1428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.133+0000 7f36008ab640 1 -- 172.21.15.64:0/847279716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc1e7260 con 0x7f35fc147b50 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.134+0000 7f36008ab640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc147b50 0x7f35fc1e6d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35f002b9e0 tx=0x7f35f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.134+0000 7f35f9ffb640 1 -- 172.21.15.64:0/847279716 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0048070 con 0x7f35fc147b50 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.134+0000 7f35f9ffb640 1 -- 172.21.15.64:0/847279716 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0030660 con 0x7f35fc147b50 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.134+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc1e7530 con 0x7f35fc147b50 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.134+0000 7f35f9ffb640 1 -- 172.21.15.64:0/847279716 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0034040 con 0x7f35fc147b50 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.134+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc143360 con 0x7f35fc147b50 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.135+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc062c90 con 0x7f35fc147b50 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.135+0000 7f35f9ffb640 1 -- 172.21.15.64:0/847279716 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f35f004b070 con 0x7f35fc147b50 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.136+0000 7f35f9ffb640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f35e806b4d0 0x7f35e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.136+0000 7f35f9ffb640 1 -- 172.21.15.64:0/847279716 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35f00abff0 con 0x7f35fc147b50 2024-03-07T07:19:19.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.136+0000 7f36010ac640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f35e806b4d0 0x7f35e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.136+0000 7f36010ac640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f35e806b4d0 0x7f35e806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009b40 tx=0x7f35f4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.140+0000 7f35f9ffb640 1 -- 172.21.15.64:0/847279716 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35f007a9d0 con 0x7f35fc147b50 2024-03-07T07:19:19.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.322+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35fc160950 con 0x7f35fc147b50 2024-03-07T07:19:19.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.323+0000 7f35f9ffb640 1 -- 172.21.15.64:0/847279716 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f35f007ea70 con 0x7f35fc147b50 2024-03-07T07:19:19.324 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:19.324 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f35e806b4d0 msgr2=0x7f35e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f35e806b4d0 0x7f35e806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009b40 tx=0x7f35f4009190 comp rx=0 tx=0).stop 2024-03-07T07:19:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc147b50 msgr2=0x7f35fc1e6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc147b50 0x7f35fc1e6d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35f002b9e0 tx=0x7f35f0004730 comp rx=0 tx=0).stop 2024-03-07T07:19:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 shutdown_connections 2024-03-07T07:19:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f35e806b4d0 0x7f35e806d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35fc147b50 0x7f35fc1e6d20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f35fc15f330 0x7f35fc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 --2- 172.21.15.64:0/847279716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35fc0b2ad0 0x7f35fc1428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 >> 172.21.15.64:0/847279716 conn(0x7f35fc06b040 msgr2=0x7f35fc071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:19.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 shutdown_connections 2024-03-07T07:19:19.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.325+0000 7f3603310640 1 -- 172.21.15.64:0/847279716 wait complete. 2024-03-07T07:19:19.337 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-07T07:19:19.337 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:19:19.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.437+0000 7f9721ad0640 1 Processor -- start 2024-03-07T07:19:19.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.437+0000 7f9721ad0640 1 -- start start 2024-03-07T07:19:19.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f9721ad0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f971c0b5dd0 0x7f971c144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f9721ad0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f9721ad0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971c162c80 con 0x7f971c0b43f0 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f9721ad0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c141700 con 0x7f971c0b5dd0 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f9721ad0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c071dc0 con 0x7f971c071410 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f971c0b5dd0 0x7f971c144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971bfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f971c0b5dd0 0x7f971c144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:56622/0 (socket says 172.21.15.64:56622) 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971b7fe640 1 -- 172.21.15.64:0/787874924 learned_addr learned my addr 172.21.15.64:0/787874924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971bfff640 1 -- 172.21.15.64:0/787874924 >> v2:172.21.15.120:3300/0 conn(0x7f971c0b5dd0 msgr2=0x7f971c144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971bfff640 1 --2- 172.21.15.64:0/787874924 >> v2:172.21.15.120:3300/0 conn(0x7f971c0b5dd0 0x7f971c144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971bfff640 1 -- 172.21.15.64:0/787874924 >> v1:172.21.15.120:6789/0 conn(0x7f971c0b43f0 legacy=0x7f971c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971bfff640 1 -- 172.21.15.64:0/787874924 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c0b4bd0 con 0x7f971c071410 2024-03-07T07:19:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971a7fc640 1 -- 172.21.15.64:0/787874924 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390045147 0 0) 0x7f971c162c80 con 0x7f971c0b43f0 2024-03-07T07:19:19.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.438+0000 7f971b7fe640 1 --2- 172.21.15.64:0/787874924 >> v2:172.21.15.120:3300/0 conn(0x7f971c0b5dd0 0x7f971c144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.439+0000 7f971bfff640 1 --2- 172.21.15.64:0/787874924 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c071880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9714009d90 tx=0x7f971402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4232c6951e4cf092 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.439+0000 7f971a7fc640 1 -- 172.21.15.64:0/787874924 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714039070 con 0x7f971c071410 2024-03-07T07:19:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.439+0000 7f971a7fc640 1 -- 172.21.15.64:0/787874924 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971402cbc0 con 0x7f971c071410 2024-03-07T07:19:19.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.439+0000 7f971a7fc640 1 -- 172.21.15.64:0/787874924 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714034440 con 0x7f971c071410 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 -- 172.21.15.64:0/787874924 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 msgr2=0x7f971c071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 --2- 172.21.15.64:0/787874924 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c071880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9714009d90 tx=0x7f971402be10 comp rx=0 tx=0).stop 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 -- 172.21.15.64:0/787874924 shutdown_connections 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 --2- 172.21.15.64:0/787874924 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c071880 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 --2- 172.21.15.64:0/787874924 >> v2:172.21.15.120:3300/0 conn(0x7f971c0b5dd0 0x7f971c144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 -- 172.21.15.64:0/787874924 >> 172.21.15.64:0/787874924 conn(0x7f971c0d0e70 msgr2=0x7f971c0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 -- 172.21.15.64:0/787874924 shutdown_connections 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.440+0000 7f9721ad0640 1 -- 172.21.15.64:0/787874924 wait complete. 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.441+0000 7f9721ad0640 1 Processor -- start 2024-03-07T07:19:19.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.441+0000 7f9721ad0640 1 -- start start 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f9721ad0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c14ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f9721ad0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f971c0b43f0 0x7f971c15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f9721ad0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c0b5dd0 0x7f971c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f9721ad0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0b5190 con 0x7f971c0b5dd0 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f9721ad0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c161e00 con 0x7f971c0b43f0 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f9721ad0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c162050 con 0x7f971c071410 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c0b5dd0 0x7f971c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c14ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c14ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:55802/0 (socket says 172.21.15.64:55802) 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c0b5dd0 0x7f971c15f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40766/0 (socket says 172.21.15.64:40766) 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971b7fe640 1 -- 172.21.15.64:0/2764233809 learned_addr learned my addr 172.21.15.64:0/2764233809 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971affd640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f971c0b43f0 0x7f971c15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971bfff640 1 -- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 msgr2=0x7f971c14ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971bfff640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c14ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971bfff640 1 -- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f971c0b43f0 msgr2=0x7f971c15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971bfff640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f971c0b43f0 0x7f971c15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.442+0000 7f971bfff640 1 -- 172.21.15.64:0/2764233809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c1f77b0 con 0x7f971c0b5dd0 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.443+0000 7f971bfff640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c0b5dd0 0x7f971c15f9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9714033000 tx=0x7f9714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.443+0000 7f9703fff640 1 -- 172.21.15.64:0/2764233809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714044020 con 0x7f971c0b5dd0 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.443+0000 7f9703fff640 1 -- 172.21.15.64:0/2764233809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971403a040 con 0x7f971c0b5dd0 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.443+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c1f7a90 con 0x7f971c0b5dd0 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.443+0000 7f9703fff640 1 -- 172.21.15.64:0/2764233809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714039050 con 0x7f971c0b5dd0 2024-03-07T07:19:19.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.443+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c1f8050 con 0x7f971c0b5dd0 2024-03-07T07:19:19.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.444+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c062c90 con 0x7f971c0b5dd0 2024-03-07T07:19:19.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.444+0000 7f9703fff640 1 -- 172.21.15.64:0/2764233809 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9714003bd0 con 0x7f971c0b5dd0 2024-03-07T07:19:19.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.445+0000 7f9703fff640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f970806b500 0x7f970806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.445+0000 7f9703fff640 1 -- 172.21.15.64:0/2764233809 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97140ac560 con 0x7f971c0b5dd0 2024-03-07T07:19:19.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.445+0000 7f971b7fe640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f970806b500 0x7f970806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.445+0000 7f971b7fe640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f970806b500 0x7f970806d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f970c009630 tx=0x7f970c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.449+0000 7f9703fff640 1 -- 172.21.15.64:0/2764233809 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f971404e060 con 0x7f971c0b5dd0 2024-03-07T07:19:19.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.626+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f971c0b8a10 con 0x7f970806b500 2024-03-07T07:19:19.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.632+0000 7f9703fff640 1 -- 172.21.15.64:0/2764233809 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237079 (secure 0 0 0) 0x7f971c0b8a10 con 0x7f970806b500 2024-03-07T07:19:19.692 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:19:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f970806b500 msgr2=0x7f970806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f970806b500 0x7f970806d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f970c009630 tx=0x7f970c009240 comp rx=0 tx=0).stop 2024-03-07T07:19:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c0b5dd0 msgr2=0x7f971c15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c0b5dd0 0x7f971c15f9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9714033000 tx=0x7f9714004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 shutdown_connections 2024-03-07T07:19:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f970806b500 0x7f970806d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971c0b5dd0 0x7f971c15f9f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f971c0b43f0 0x7f971c15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 --2- 172.21.15.64:0/2764233809 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f971c071410 0x7f971c14ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 >> 172.21.15.64:0/2764233809 conn(0x7f971c0d0e70 msgr2=0x7f971c0d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 shutdown_connections 2024-03-07T07:19:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.635+0000 7f9721ad0640 1 -- 172.21.15.64:0/2764233809 wait complete. 2024-03-07T07:19:19.696 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-07T07:19:18.180394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219436,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755284,"statfs":{"total":773094113280,"available":772869410816,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000897"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-07T07:19:07.015553+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:19:07.015553+0000","last_peered":"2024-03-07T07:19:07.015553+0000","last_clean":"2024-03-07T07:19:07.015553+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:19:07.015553+0000","last_undegraded":"2024-03-07T07:19:07.015553+0000","last_fullsized":"2024-03-07T07:19:07.015553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:19:19.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:19:19.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.800+0000 7f0e77f61640 1 Processor -- start 2024-03-07T07:19:19.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.801+0000 7f0e77f61640 1 -- start start 2024-03-07T07:19:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.801+0000 7f0e77f61640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e70144590 0x7f0e7015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e77f61640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7015f410 con 0x7f0e700d2d30 2024-03-07T07:19:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e77f61640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e70143ab0 con 0x7f0e70072320 2024-03-07T07:19:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e77f61640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e700b9640 con 0x7f0e70144590 2024-03-07T07:19:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e764fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e70144590 0x7f0e7015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e75cfd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0e700d2d30 0x7f0e70071c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37426/0 (socket says 172.21.15.64:37426) 2024-03-07T07:19:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e764fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e70144590 0x7f0e7015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40774/0 (socket says 172.21.15.64:40774) 2024-03-07T07:19:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e75cfd640 1 -- 172.21.15.64:0/336170216 learned_addr learned my addr 172.21.15.64:0/336170216 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e764fe640 1 -- 172.21.15.64:0/336170216 >> v1:172.21.15.120:6789/0 conn(0x7f0e70072320 legacy=0x7f0e7015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e764fe640 1 -- 172.21.15.64:0/336170216 >> v1:172.21.15.64:6789/0 conn(0x7f0e700d2d30 legacy=0x7f0e70071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e74cfb640 1 -- 172.21.15.64:0/336170216 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738196858 0 0) 0x7f0e7015f410 con 0x7f0e700d2d30 2024-03-07T07:19:19.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.802+0000 7f0e764fe640 1 -- 172.21.15.64:0/336170216 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e70143d90 con 0x7f0e70144590 2024-03-07T07:19:19.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e764fe640 1 --2- 172.21.15.64:0/336170216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70144590 0x7f0e7015ee10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009d90 tx=0x7f0e6802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc82dae413ddf1e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e74cfb640 1 -- 172.21.15.64:0/336170216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68039070 con 0x7f0e70144590 2024-03-07T07:19:19.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e74cfb640 1 -- 172.21.15.64:0/336170216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6802cbd0 con 0x7f0e70144590 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e74cfb640 1 -- 172.21.15.64:0/336170216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68034440 con 0x7f0e70144590 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e77f61640 1 -- 172.21.15.64:0/336170216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70144590 msgr2=0x7f0e7015ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e77f61640 1 --2- 172.21.15.64:0/336170216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70144590 0x7f0e7015ee10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009d90 tx=0x7f0e6802be20 comp rx=0 tx=0).stop 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e77f61640 1 -- 172.21.15.64:0/336170216 shutdown_connections 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e77f61640 1 --2- 172.21.15.64:0/336170216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70144590 0x7f0e7015ee10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.803+0000 7f0e77f61640 1 -- 172.21.15.64:0/336170216 >> 172.21.15.64:0/336170216 conn(0x7f0e700707f0 msgr2=0x7f0e700b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.804+0000 7f0e77f61640 1 -- 172.21.15.64:0/336170216 shutdown_connections 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.804+0000 7f0e77f61640 1 -- 172.21.15.64:0/336170216 wait complete. 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 Processor -- start 2024-03-07T07:19:19.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 -- start start 2024-03-07T07:19:19.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70072320 0x7f0e7013b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e700d2d30 0x7f0e701463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e70144590 0x7f0e700b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70169000 con 0x7f0e70072320 2024-03-07T07:19:19.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70168e80 con 0x7f0e70144590 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e77f61640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70169180 con 0x7f0e700d2d30 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e75cfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70072320 0x7f0e7013b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e75cfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70072320 0x7f0e7013b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40782/0 (socket says 172.21.15.64:40782) 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.805+0000 7f0e75cfd640 1 -- 172.21.15.64:0/3328996363 learned_addr learned my addr 172.21.15.64:0/3328996363 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e754fc640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e700d2d30 0x7f0e701463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e764fe640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e70144590 0x7f0e700b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e75cfd640 1 -- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e700d2d30 msgr2=0x7f0e701463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e75cfd640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e700d2d30 0x7f0e701463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e75cfd640 1 -- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e70144590 msgr2=0x7f0e700b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e75cfd640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e70144590 0x7f0e700b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e75cfd640 1 -- 172.21.15.64:0/3328996363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e70146df0 con 0x7f0e70072320 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e75cfd640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70072320 0x7f0e7013b7f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c3f0 tx=0x7f0e6c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e667fc640 1 -- 172.21.15.64:0/3328996363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0074e0 con 0x7f0e70072320 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e667fc640 1 -- 172.21.15.64:0/3328996363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0047e0 con 0x7f0e70072320 2024-03-07T07:19:19.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e701470d0 con 0x7f0e70072320 2024-03-07T07:19:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e667fc640 1 -- 172.21.15.64:0/3328996363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c004bd0 con 0x7f0e70072320 2024-03-07T07:19:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.806+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e70147690 con 0x7f0e70072320 2024-03-07T07:19:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.807+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e70062c90 con 0x7f0e70072320 2024-03-07T07:19:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.807+0000 7f0e667fc640 1 -- 172.21.15.64:0/3328996363 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e6c011070 con 0x7f0e70072320 2024-03-07T07:19:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.808+0000 7f0e667fc640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e5406b4d0 0x7f0e5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.808+0000 7f0e754fc640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e5406b4d0 0x7f0e5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:19.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.808+0000 7f0e667fc640 1 -- 172.21.15.64:0/3328996363 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e6c089ce0 con 0x7f0e70072320 2024-03-07T07:19:19.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.808+0000 7f0e754fc640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e5406b4d0 0x7f0e5406d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e7015ddb0 tx=0x7f0e60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:19.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.812+0000 7f0e667fc640 1 -- 172.21.15.64:0/3328996363 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e6c058510 con 0x7f0e70072320 2024-03-07T07:19:19.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.988+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e7015c400 con 0x7f0e5406b4d0 2024-03-07T07:19:19.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.994+0000 7f0e667fc640 1 -- 172.21.15.64:0/3328996363 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237079 (secure 0 0 0) 0x7f0e7015c400 con 0x7f0e5406b4d0 2024-03-07T07:19:19.997 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:19.998 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.996+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e5406b4d0 msgr2=0x7f0e5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.996+0000 7f0e77f61640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e5406b4d0 0x7f0e5406d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e7015ddb0 tx=0x7f0e60009040 comp rx=0 tx=0).stop 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.996+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70072320 msgr2=0x7f0e7013b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.996+0000 7f0e77f61640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70072320 0x7f0e7013b7f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c3f0 tx=0x7f0e6c00c8b0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.996+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 shutdown_connections 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.996+0000 7f0e77f61640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e5406b4d0 0x7f0e5406d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.997+0000 7f0e77f61640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e70144590 0x7f0e700b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.997+0000 7f0e77f61640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e700d2d30 0x7f0e701463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.997+0000 7f0e77f61640 1 --2- 172.21.15.64:0/3328996363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e70072320 0x7f0e7013b7f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.997+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 >> 172.21.15.64:0/3328996363 conn(0x7f0e700707f0 msgr2=0x7f0e70162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:20.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.997+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 shutdown_connections 2024-03-07T07:19:20.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:19.997+0000 7f0e77f61640 1 -- 172.21.15.64:0/3328996363 wait complete. 2024-03-07T07:19:20.008 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-07T07:19:18.180394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219436,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755284,"statfs":{"total":773094113280,"available":772869410816,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000897"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-07T07:19:07.015553+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:19:07.015553+0000","last_peered":"2024-03-07T07:19:07.015553+0000","last_clean":"2024-03-07T07:19:07.015553+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:19:07.015553+0000","last_undegraded":"2024-03-07T07:19:07.015553+0000","last_fullsized":"2024-03-07T07:19:07.015553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:15.273316+0000","last_change":"2024-03-07T07:19:15.273316+0000","last_active":"2024-03-07T07:19:15.273316+0000","last_peered":"2024-03-07T07:19:15.273316+0000","last_clean":"2024-03-07T07:19:15.273316+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:15.273316+0000","last_undegraded":"2024-03-07T07:19:15.273316+0000","last_fullsized":"2024-03-07T07:19:15.273316+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T07:19:14.267775+0000","last_change":"2024-03-07T07:19:14.267775+0000","last_active":"2024-03-07T07:19:14.267775+0000","last_peered":"2024-03-07T07:19:14.267775+0000","last_clean":"2024-03-07T07:19:14.267775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T07:19:14.267775+0000","last_undegraded":"2024-03-07T07:19:14.267775+0000","last_fullsized":"2024-03-07T07:19:14.267775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:19:20.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:19:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.149+0000 7fbe4ea87640 1 Processor -- start 2024-03-07T07:19:20.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.149+0000 7fbe4ea87640 1 -- start start 2024-03-07T07:19:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4ea87640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbe480aef50 0x7fbe480b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4ea87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe48071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4ea87640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4806f560 con 0x7fbe4815e050 2024-03-07T07:19:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4ea87640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4806e7a0 con 0x7fbe480aef50 2024-03-07T07:19:20.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4ea87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4806ea80 con 0x7fbe480b0a00 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4c823640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbe4815e050 0x7fbe480ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:37432/0 (socket says 172.21.15.64:37432) 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4c823640 1 -- 172.21.15.64:0/2359281109 learned_addr learned my addr 172.21.15.64:0/2359281109 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe47fff640 1 --2- 172.21.15.64:0/2359281109 >> v2:172.21.15.120:3300/0 conn(0x7fbe480aef50 0x7fbe480b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4d024640 1 --2- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe48071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe477fe640 1 -- 172.21.15.64:0/2359281109 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032673697 0 0) 0x7fbe4806f560 con 0x7fbe4815e050 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe47fff640 1 -- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 msgr2=0x7fbe48071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe47fff640 1 --2- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe48071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe47fff640 1 -- 172.21.15.64:0/2359281109 >> v1:172.21.15.64:6789/0 conn(0x7fbe4815e050 legacy=0x7fbe480ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe47fff640 1 -- 172.21.15.64:0/2359281109 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4806ed60 con 0x7fbe480aef50 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.150+0000 7fbe4d024640 1 --2- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe48071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.151+0000 7fbe47fff640 1 --2- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe480b04c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400ba40 tx=0x7fbe3402dff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6797b290123b0021 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.151+0000 7fbe477fe640 1 -- 172.21.15.64:0/2359281109 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3403b070 con 0x7fbe480aef50 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.151+0000 7fbe477fe640 1 -- 172.21.15.64:0/2359281109 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3402eda0 con 0x7fbe480aef50 2024-03-07T07:19:20.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.151+0000 7fbe477fe640 1 -- 172.21.15.64:0/2359281109 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34036660 con 0x7fbe480aef50 2024-03-07T07:19:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 -- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 msgr2=0x7fbe480b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe480b04c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400ba40 tx=0x7fbe3402dff0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 -- 172.21.15.64:0/2359281109 shutdown_connections 2024-03-07T07:19:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe48071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/2359281109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe480b04c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 -- 172.21.15.64:0/2359281109 >> 172.21.15.64:0/2359281109 conn(0x7fbe480b3120 msgr2=0x7fbe480b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:20.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 -- 172.21.15.64:0/2359281109 shutdown_connections 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.152+0000 7fbe4ea87640 1 -- 172.21.15.64:0/2359281109 wait complete. 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.153+0000 7fbe4ea87640 1 Processor -- start 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.153+0000 7fbe4ea87640 1 -- start start 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe481466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe480b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe4815e050 0x7fbe4815b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480cee60 con 0x7fbe4815e050 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480cece0 con 0x7fbe480aef50 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480afc30 con 0x7fbe480b0a00 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe4815e050 0x7fbe4815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4c823640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe481466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe480b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe4815e050 0x7fbe4815b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40796/0 (socket says 172.21.15.64:40796) 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 -- 172.21.15.64:0/1056946406 learned_addr learned my addr 172.21.15.64:0/1056946406 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 -- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 msgr2=0x7fbe480b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe480b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 -- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 msgr2=0x7fbe481466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe481466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 -- 172.21.15.64:0/1056946406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe481e3420 con 0x7fbe4815e050 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe47fff640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe480b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4c823640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe481466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4d024640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe4815e050 0x7fbe4815b760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000efc0 tx=0x7fbe40009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe457fa640 1 -- 172.21.15.64:0/1056946406 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40018030 con 0x7fbe4815e050 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe457fa640 1 -- 172.21.15.64:0/1056946406 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4000cde0 con 0x7fbe4815e050 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe481e3700 con 0x7fbe4815e050 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe457fa640 1 -- 172.21.15.64:0/1056946406 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40003bb0 con 0x7fbe4815e050 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.154+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe481e3cc0 con 0x7fbe4815e050 2024-03-07T07:19:20.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.155+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe48062c90 con 0x7fbe4815e050 2024-03-07T07:19:20.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.156+0000 7fbe457fa640 1 -- 172.21.15.64:0/1056946406 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbe4001b070 con 0x7fbe4815e050 2024-03-07T07:19:20.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.156+0000 7fbe457fa640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbe2c06b500 0x7fbe2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.156+0000 7fbe457fa640 1 -- 172.21.15.64:0/1056946406 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe4008ab00 con 0x7fbe4815e050 2024-03-07T07:19:20.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.156+0000 7fbe4c823640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbe2c06b500 0x7fbe2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.157+0000 7fbe4c823640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbe2c06b500 0x7fbe2c06d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007fd0 tx=0x7fbe3c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.161+0000 7fbe457fa640 1 -- 172.21.15.64:0/1056946406 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe40010040 con 0x7fbe4815e050 2024-03-07T07:19:20.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.339+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe480744d0 con 0x7fbe2c06b500 2024-03-07T07:19:20.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.346+0000 7fbe457fa640 1 -- 172.21.15.64:0/1056946406 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251948 (secure 0 0 0) 0x7fbe480744d0 con 0x7fbe2c06b500 2024-03-07T07:19:20.351 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:20.352 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbe2c06b500 msgr2=0x7fbe2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbe2c06b500 0x7fbe2c06d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007fd0 tx=0x7fbe3c007de0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe4815e050 msgr2=0x7fbe4815b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe4815e050 0x7fbe4815b760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000efc0 tx=0x7fbe40009f90 comp rx=0 tx=0).stop 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 shutdown_connections 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbe2c06b500 0x7fbe2c06d9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe4815e050 0x7fbe4815b760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbe480b0a00 0x7fbe480b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 --2- 172.21.15.64:0/1056946406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe480aef50 0x7fbe481466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 >> 172.21.15.64:0/1056946406 conn(0x7fbe480b3120 msgr2=0x7fbe480b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 shutdown_connections 2024-03-07T07:19:20.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.349+0000 7fbe4ea87640 1 -- 172.21.15.64:0/1056946406 wait complete. 2024-03-07T07:19:20.361 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-07T07:19:20.180659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.906365"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069031+0000","last_change":"2024-03-07T07:19:16.302696+0000","last_active":"2024-03-07T07:19:18.069031+0000","last_peered":"2024-03-07T07:19:18.069031+0000","last_clean":"2024-03-07T07:19:18.069031+0000","last_became_active":"2024-03-07T07:19:16.302389+0000","last_became_peered":"2024-03-07T07:19:16.302389+0000","last_unstale":"2024-03-07T07:19:18.069031+0000","last_undegraded":"2024-03-07T07:19:18.069031+0000","last_fullsized":"2024-03-07T07:19:18.069031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294255+0000","last_change":"2024-03-07T07:19:16.303217+0000","last_active":"2024-03-07T07:19:17.294255+0000","last_peered":"2024-03-07T07:19:17.294255+0000","last_clean":"2024-03-07T07:19:17.294255+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:17.294255+0000","last_undegraded":"2024-03-07T07:19:17.294255+0000","last_fullsized":"2024-03-07T07:19:17.294255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870246+0000","last_change":"2024-03-07T07:19:16.292858+0000","last_active":"2024-03-07T07:19:17.870246+0000","last_peered":"2024-03-07T07:19:17.870246+0000","last_clean":"2024-03-07T07:19:17.870246+0000","last_became_active":"2024-03-07T07:19:16.292593+0000","last_became_peered":"2024-03-07T07:19:16.292593+0000","last_unstale":"2024-03-07T07:19:17.870246+0000","last_undegraded":"2024-03-07T07:19:17.870246+0000","last_fullsized":"2024-03-07T07:19:17.870246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417391+0000","last_change":"2024-03-07T07:19:16.310590+0000","last_active":"2024-03-07T07:19:17.417391+0000","last_peered":"2024-03-07T07:19:17.417391+0000","last_clean":"2024-03-07T07:19:17.417391+0000","last_became_active":"2024-03-07T07:19:16.310298+0000","last_became_peered":"2024-03-07T07:19:16.310298+0000","last_unstale":"2024-03-07T07:19:17.417391+0000","last_undegraded":"2024-03-07T07:19:17.417391+0000","last_fullsized":"2024-03-07T07:19:17.417391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069132+0000","last_change":"2024-03-07T07:19:16.299586+0000","last_active":"2024-03-07T07:19:18.069132+0000","last_peered":"2024-03-07T07:19:18.069132+0000","last_clean":"2024-03-07T07:19:18.069132+0000","last_became_active":"2024-03-07T07:19:16.299265+0000","last_became_peered":"2024-03-07T07:19:16.299265+0000","last_unstale":"2024-03-07T07:19:18.069132+0000","last_undegraded":"2024-03-07T07:19:18.069132+0000","last_fullsized":"2024-03-07T07:19:18.069132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417301+0000","last_change":"2024-03-07T07:19:16.312616+0000","last_active":"2024-03-07T07:19:17.417301+0000","last_peered":"2024-03-07T07:19:17.417301+0000","last_clean":"2024-03-07T07:19:17.417301+0000","last_became_active":"2024-03-07T07:19:16.312339+0000","last_became_peered":"2024-03-07T07:19:16.312339+0000","last_unstale":"2024-03-07T07:19:17.417301+0000","last_undegraded":"2024-03-07T07:19:17.417301+0000","last_fullsized":"2024-03-07T07:19:17.417301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068749+0000","last_change":"2024-03-07T07:19:16.302351+0000","last_active":"2024-03-07T07:19:18.068749+0000","last_peered":"2024-03-07T07:19:18.068749+0000","last_clean":"2024-03-07T07:19:18.068749+0000","last_became_active":"2024-03-07T07:19:16.302071+0000","last_became_peered":"2024-03-07T07:19:16.302071+0000","last_unstale":"2024-03-07T07:19:18.068749+0000","last_undegraded":"2024-03-07T07:19:18.068749+0000","last_fullsized":"2024-03-07T07:19:18.068749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009417+0000","last_change":"2024-03-07T07:19:16.310400+0000","last_active":"2024-03-07T07:19:18.009417+0000","last_peered":"2024-03-07T07:19:18.009417+0000","last_clean":"2024-03-07T07:19:18.009417+0000","last_became_active":"2024-03-07T07:19:16.310066+0000","last_became_peered":"2024-03-07T07:19:16.310066+0000","last_unstale":"2024-03-07T07:19:18.009417+0000","last_undegraded":"2024-03-07T07:19:18.009417+0000","last_fullsized":"2024-03-07T07:19:18.009417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069013+0000","last_change":"2024-03-07T07:19:16.307888+0000","last_active":"2024-03-07T07:19:18.069013+0000","last_peered":"2024-03-07T07:19:18.069013+0000","last_clean":"2024-03-07T07:19:18.069013+0000","last_became_active":"2024-03-07T07:19:16.306871+0000","last_became_peered":"2024-03-07T07:19:16.306871+0000","last_unstale":"2024-03-07T07:19:18.069013+0000","last_undegraded":"2024-03-07T07:19:18.069013+0000","last_fullsized":"2024-03-07T07:19:18.069013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296721+0000","last_change":"2024-03-07T07:19:16.306115+0000","last_active":"2024-03-07T07:19:17.296721+0000","last_peered":"2024-03-07T07:19:17.296721+0000","last_clean":"2024-03-07T07:19:17.296721+0000","last_became_active":"2024-03-07T07:19:16.305591+0000","last_became_peered":"2024-03-07T07:19:16.305591+0000","last_unstale":"2024-03-07T07:19:17.296721+0000","last_undegraded":"2024-03-07T07:19:17.296721+0000","last_fullsized":"2024-03-07T07:19:17.296721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296408+0000","last_change":"2024-03-07T07:19:16.305713+0000","last_active":"2024-03-07T07:19:17.296408+0000","last_peered":"2024-03-07T07:19:17.296408+0000","last_clean":"2024-03-07T07:19:17.296408+0000","last_became_active":"2024-03-07T07:19:16.305408+0000","last_became_peered":"2024-03-07T07:19:16.305408+0000","last_unstale":"2024-03-07T07:19:17.296408+0000","last_undegraded":"2024-03-07T07:19:17.296408+0000","last_fullsized":"2024-03-07T07:19:17.296408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297262+0000","last_change":"2024-03-07T07:19:16.301659+0000","last_active":"2024-03-07T07:19:17.297262+0000","last_peered":"2024-03-07T07:19:17.297262+0000","last_clean":"2024-03-07T07:19:17.297262+0000","last_became_active":"2024-03-07T07:19:16.301033+0000","last_became_peered":"2024-03-07T07:19:16.301033+0000","last_unstale":"2024-03-07T07:19:17.297262+0000","last_undegraded":"2024-03-07T07:19:17.297262+0000","last_fullsized":"2024-03-07T07:19:17.297262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:34.845258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009648+0000","last_change":"2024-03-07T07:19:16.296414+0000","last_active":"2024-03-07T07:19:18.009648+0000","last_peered":"2024-03-07T07:19:18.009648+0000","last_clean":"2024-03-07T07:19:18.009648+0000","last_became_active":"2024-03-07T07:19:16.296078+0000","last_became_peered":"2024-03-07T07:19:16.296078+0000","last_unstale":"2024-03-07T07:19:18.009648+0000","last_undegraded":"2024-03-07T07:19:18.009648+0000","last_fullsized":"2024-03-07T07:19:18.009648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870192+0000","last_change":"2024-03-07T07:19:16.298815+0000","last_active":"2024-03-07T07:19:17.870192+0000","last_peered":"2024-03-07T07:19:17.870192+0000","last_clean":"2024-03-07T07:19:17.870192+0000","last_became_active":"2024-03-07T07:19:16.298560+0000","last_became_peered":"2024-03-07T07:19:16.298560+0000","last_unstale":"2024-03-07T07:19:17.870192+0000","last_undegraded":"2024-03-07T07:19:17.870192+0000","last_fullsized":"2024-03-07T07:19:17.870192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:14:50.701779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293979+0000","last_change":"2024-03-07T07:19:16.304915+0000","last_active":"2024-03-07T07:19:17.293979+0000","last_peered":"2024-03-07T07:19:17.293979+0000","last_clean":"2024-03-07T07:19:17.293979+0000","last_became_active":"2024-03-07T07:19:16.304581+0000","last_became_peered":"2024-03-07T07:19:16.304581+0000","last_unstale":"2024-03-07T07:19:17.293979+0000","last_undegraded":"2024-03-07T07:19:17.293979+0000","last_fullsized":"2024-03-07T07:19:17.293979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417242+0000","last_change":"2024-03-07T07:19:16.312660+0000","last_active":"2024-03-07T07:19:17.417242+0000","last_peered":"2024-03-07T07:19:17.417242+0000","last_clean":"2024-03-07T07:19:17.417242+0000","last_became_active":"2024-03-07T07:19:16.312356+0000","last_became_peered":"2024-03-07T07:19:16.312356+0000","last_unstale":"2024-03-07T07:19:17.417242+0000","last_undegraded":"2024-03-07T07:19:17.417242+0000","last_fullsized":"2024-03-07T07:19:17.417242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417697+0000","last_change":"2024-03-07T07:19:16.311126+0000","last_active":"2024-03-07T07:19:17.417697+0000","last_peered":"2024-03-07T07:19:17.417697+0000","last_clean":"2024-03-07T07:19:17.417697+0000","last_became_active":"2024-03-07T07:19:16.310806+0000","last_became_peered":"2024-03-07T07:19:16.310806+0000","last_unstale":"2024-03-07T07:19:17.417697+0000","last_undegraded":"2024-03-07T07:19:17.417697+0000","last_fullsized":"2024-03-07T07:19:17.417697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068899+0000","last_change":"2024-03-07T07:19:16.296921+0000","last_active":"2024-03-07T07:19:18.068899+0000","last_peered":"2024-03-07T07:19:18.068899+0000","last_clean":"2024-03-07T07:19:18.068899+0000","last_became_active":"2024-03-07T07:19:16.296657+0000","last_became_peered":"2024-03-07T07:19:16.296657+0000","last_unstale":"2024-03-07T07:19:18.068899+0000","last_undegraded":"2024-03-07T07:19:18.068899+0000","last_fullsized":"2024-03-07T07:19:18.068899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069320+0000","last_change":"2024-03-07T07:19:16.301924+0000","last_active":"2024-03-07T07:19:18.069320+0000","last_peered":"2024-03-07T07:19:18.069320+0000","last_clean":"2024-03-07T07:19:18.069320+0000","last_became_active":"2024-03-07T07:19:16.301657+0000","last_became_peered":"2024-03-07T07:19:16.301657+0000","last_unstale":"2024-03-07T07:19:18.069320+0000","last_undegraded":"2024-03-07T07:19:18.069320+0000","last_fullsized":"2024-03-07T07:19:18.069320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:16:48.547021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296490+0000","last_change":"2024-03-07T07:19:16.300281+0000","last_active":"2024-03-07T07:19:17.296490+0000","last_peered":"2024-03-07T07:19:17.296490+0000","last_clean":"2024-03-07T07:19:17.296490+0000","last_became_active":"2024-03-07T07:19:16.299775+0000","last_became_peered":"2024-03-07T07:19:16.299775+0000","last_unstale":"2024-03-07T07:19:17.296490+0000","last_undegraded":"2024-03-07T07:19:17.296490+0000","last_fullsized":"2024-03-07T07:19:17.296490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870510+0000","last_change":"2024-03-07T07:19:16.296327+0000","last_active":"2024-03-07T07:19:17.870510+0000","last_peered":"2024-03-07T07:19:17.870510+0000","last_clean":"2024-03-07T07:19:17.870510+0000","last_became_active":"2024-03-07T07:19:16.295959+0000","last_became_peered":"2024-03-07T07:19:16.295959+0000","last_unstale":"2024-03-07T07:19:17.870510+0000","last_undegraded":"2024-03-07T07:19:17.870510+0000","last_fullsized":"2024-03-07T07:19:17.870510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417371+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:17.417371+0000","last_peered":"2024-03-07T07:19:17.417371+0000","last_clean":"2024-03-07T07:19:17.417371+0000","last_became_active":"2024-03-07T07:19:16.307389+0000","last_became_peered":"2024-03-07T07:19:16.307389+0000","last_unstale":"2024-03-07T07:19:17.417371+0000","last_undegraded":"2024-03-07T07:19:17.417371+0000","last_fullsized":"2024-03-07T07:19:17.417371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009458+0000","last_change":"2024-03-07T07:19:16.295700+0000","last_active":"2024-03-07T07:19:18.009458+0000","last_peered":"2024-03-07T07:19:18.009458+0000","last_clean":"2024-03-07T07:19:18.009458+0000","last_became_active":"2024-03-07T07:19:16.295428+0000","last_became_peered":"2024-03-07T07:19:16.295428+0000","last_unstale":"2024-03-07T07:19:18.009458+0000","last_undegraded":"2024-03-07T07:19:18.009458+0000","last_fullsized":"2024-03-07T07:19:18.009458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417302+0000","last_change":"2024-03-07T07:19:16.305382+0000","last_active":"2024-03-07T07:19:17.417302+0000","last_peered":"2024-03-07T07:19:17.417302+0000","last_clean":"2024-03-07T07:19:17.417302+0000","last_became_active":"2024-03-07T07:19:16.304697+0000","last_became_peered":"2024-03-07T07:19:16.304697+0000","last_unstale":"2024-03-07T07:19:17.417302+0000","last_undegraded":"2024-03-07T07:19:17.417302+0000","last_fullsized":"2024-03-07T07:19:17.417302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009833+0000","last_change":"2024-03-07T07:19:16.295728+0000","last_active":"2024-03-07T07:19:18.009833+0000","last_peered":"2024-03-07T07:19:18.009833+0000","last_clean":"2024-03-07T07:19:18.009833+0000","last_became_active":"2024-03-07T07:19:16.295472+0000","last_became_peered":"2024-03-07T07:19:16.295472+0000","last_unstale":"2024-03-07T07:19:18.009833+0000","last_undegraded":"2024-03-07T07:19:18.009833+0000","last_fullsized":"2024-03-07T07:19:18.009833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296848+0000","last_change":"2024-03-07T07:19:16.297196+0000","last_active":"2024-03-07T07:19:17.296848+0000","last_peered":"2024-03-07T07:19:17.296848+0000","last_clean":"2024-03-07T07:19:17.296848+0000","last_became_active":"2024-03-07T07:19:16.296727+0000","last_became_peered":"2024-03-07T07:19:16.296727+0000","last_unstale":"2024-03-07T07:19:17.296848+0000","last_undegraded":"2024-03-07T07:19:17.296848+0000","last_fullsized":"2024-03-07T07:19:17.296848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294209+0000","last_change":"2024-03-07T07:19:16.299086+0000","last_active":"2024-03-07T07:19:17.294209+0000","last_peered":"2024-03-07T07:19:17.294209+0000","last_clean":"2024-03-07T07:19:17.294209+0000","last_became_active":"2024-03-07T07:19:16.298819+0000","last_became_peered":"2024-03-07T07:19:16.298819+0000","last_unstale":"2024-03-07T07:19:17.294209+0000","last_undegraded":"2024-03-07T07:19:17.294209+0000","last_fullsized":"2024-03-07T07:19:17.294209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009756+0000","last_change":"2024-03-07T07:19:16.299038+0000","last_active":"2024-03-07T07:19:18.009756+0000","last_peered":"2024-03-07T07:19:18.009756+0000","last_clean":"2024-03-07T07:19:18.009756+0000","last_became_active":"2024-03-07T07:19:16.298748+0000","last_became_peered":"2024-03-07T07:19:16.298748+0000","last_unstale":"2024-03-07T07:19:18.009756+0000","last_undegraded":"2024-03-07T07:19:18.009756+0000","last_fullsized":"2024-03-07T07:19:18.009756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870540+0000","last_change":"2024-03-07T07:19:16.293023+0000","last_active":"2024-03-07T07:19:17.870540+0000","last_peered":"2024-03-07T07:19:17.870540+0000","last_clean":"2024-03-07T07:19:17.870540+0000","last_became_active":"2024-03-07T07:19:16.292670+0000","last_became_peered":"2024-03-07T07:19:16.292670+0000","last_unstale":"2024-03-07T07:19:17.870540+0000","last_undegraded":"2024-03-07T07:19:17.870540+0000","last_fullsized":"2024-03-07T07:19:17.870540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417541+0000","last_change":"2024-03-07T07:19:16.310502+0000","last_active":"2024-03-07T07:19:17.417541+0000","last_peered":"2024-03-07T07:19:17.417541+0000","last_clean":"2024-03-07T07:19:17.417541+0000","last_became_active":"2024-03-07T07:19:16.309883+0000","last_became_peered":"2024-03-07T07:19:16.309883+0000","last_unstale":"2024-03-07T07:19:17.417541+0000","last_undegraded":"2024-03-07T07:19:17.417541+0000","last_fullsized":"2024-03-07T07:19:17.417541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068772+0000","last_change":"2024-03-07T07:19:16.306042+0000","last_active":"2024-03-07T07:19:18.068772+0000","last_peered":"2024-03-07T07:19:18.068772+0000","last_clean":"2024-03-07T07:19:18.068772+0000","last_became_active":"2024-03-07T07:19:16.304954+0000","last_became_peered":"2024-03-07T07:19:16.304954+0000","last_unstale":"2024-03-07T07:19:18.068772+0000","last_undegraded":"2024-03-07T07:19:18.068772+0000","last_fullsized":"2024-03-07T07:19:18.068772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296496+0000","last_change":"2024-03-07T07:19:16.300717+0000","last_active":"2024-03-07T07:19:17.296496+0000","last_peered":"2024-03-07T07:19:17.296496+0000","last_clean":"2024-03-07T07:19:17.296496+0000","last_became_active":"2024-03-07T07:19:16.300170+0000","last_became_peered":"2024-03-07T07:19:16.300170+0000","last_unstale":"2024-03-07T07:19:17.296496+0000","last_undegraded":"2024-03-07T07:19:17.296496+0000","last_fullsized":"2024-03-07T07:19:17.296496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297170+0000","last_change":"2024-03-07T07:19:16.299268+0000","last_active":"2024-03-07T07:19:17.297170+0000","last_peered":"2024-03-07T07:19:17.297170+0000","last_clean":"2024-03-07T07:19:17.297170+0000","last_became_active":"2024-03-07T07:19:16.298987+0000","last_became_peered":"2024-03-07T07:19:16.298987+0000","last_unstale":"2024-03-07T07:19:17.297170+0000","last_undegraded":"2024-03-07T07:19:17.297170+0000","last_fullsized":"2024-03-07T07:19:17.297170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:19.940197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009780+0000","last_change":"2024-03-07T07:19:15.298629+0000","last_active":"2024-03-07T07:19:18.009780+0000","last_peered":"2024-03-07T07:19:18.009780+0000","last_clean":"2024-03-07T07:19:18.009780+0000","last_became_active":"2024-03-07T07:19:15.298338+0000","last_became_peered":"2024-03-07T07:19:15.298338+0000","last_unstale":"2024-03-07T07:19:18.009780+0000","last_undegraded":"2024-03-07T07:19:18.009780+0000","last_fullsized":"2024-03-07T07:19:18.009780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870658+0000","last_change":"2024-03-07T07:19:16.299764+0000","last_active":"2024-03-07T07:19:17.870658+0000","last_peered":"2024-03-07T07:19:17.870658+0000","last_clean":"2024-03-07T07:19:17.870658+0000","last_became_active":"2024-03-07T07:19:16.299493+0000","last_became_peered":"2024-03-07T07:19:16.299493+0000","last_unstale":"2024-03-07T07:19:17.870658+0000","last_undegraded":"2024-03-07T07:19:17.870658+0000","last_fullsized":"2024-03-07T07:19:17.870658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296626+0000","last_change":"2024-03-07T07:19:15.285743+0000","last_active":"2024-03-07T07:19:17.296626+0000","last_peered":"2024-03-07T07:19:17.296626+0000","last_clean":"2024-03-07T07:19:17.296626+0000","last_became_active":"2024-03-07T07:19:15.285484+0000","last_became_peered":"2024-03-07T07:19:15.285484+0000","last_unstale":"2024-03-07T07:19:17.296626+0000","last_undegraded":"2024-03-07T07:19:17.296626+0000","last_fullsized":"2024-03-07T07:19:17.296626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009544+0000","last_change":"2024-03-07T07:19:15.294301+0000","last_active":"2024-03-07T07:19:18.009544+0000","last_peered":"2024-03-07T07:19:18.009544+0000","last_clean":"2024-03-07T07:19:18.009544+0000","last_became_active":"2024-03-07T07:19:15.294023+0000","last_became_peered":"2024-03-07T07:19:15.294023+0000","last_unstale":"2024-03-07T07:19:18.009544+0000","last_undegraded":"2024-03-07T07:19:18.009544+0000","last_fullsized":"2024-03-07T07:19:18.009544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009524+0000","last_change":"2024-03-07T07:19:16.299031+0000","last_active":"2024-03-07T07:19:18.009524+0000","last_peered":"2024-03-07T07:19:18.009524+0000","last_clean":"2024-03-07T07:19:18.009524+0000","last_became_active":"2024-03-07T07:19:16.298760+0000","last_became_peered":"2024-03-07T07:19:16.298760+0000","last_unstale":"2024-03-07T07:19:18.009524+0000","last_undegraded":"2024-03-07T07:19:18.009524+0000","last_fullsized":"2024-03-07T07:19:18.009524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417339+0000","last_change":"2024-03-07T07:19:15.292620+0000","last_active":"2024-03-07T07:19:17.417339+0000","last_peered":"2024-03-07T07:19:17.417339+0000","last_clean":"2024-03-07T07:19:17.417339+0000","last_became_active":"2024-03-07T07:19:15.292337+0000","last_became_peered":"2024-03-07T07:19:15.292337+0000","last_unstale":"2024-03-07T07:19:17.417339+0000","last_undegraded":"2024-03-07T07:19:17.417339+0000","last_fullsized":"2024-03-07T07:19:17.417339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293989+0000","last_change":"2024-03-07T07:19:16.297970+0000","last_active":"2024-03-07T07:19:17.293989+0000","last_peered":"2024-03-07T07:19:17.293989+0000","last_clean":"2024-03-07T07:19:17.293989+0000","last_became_active":"2024-03-07T07:19:16.297661+0000","last_became_peered":"2024-03-07T07:19:16.297661+0000","last_unstale":"2024-03-07T07:19:17.293989+0000","last_undegraded":"2024-03-07T07:19:17.293989+0000","last_fullsized":"2024-03-07T07:19:17.293989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296514+0000","last_change":"2024-03-07T07:19:15.295817+0000","last_active":"2024-03-07T07:19:17.296514+0000","last_peered":"2024-03-07T07:19:17.296514+0000","last_clean":"2024-03-07T07:19:17.296514+0000","last_became_active":"2024-03-07T07:19:15.295538+0000","last_became_peered":"2024-03-07T07:19:15.295538+0000","last_unstale":"2024-03-07T07:19:17.296514+0000","last_undegraded":"2024-03-07T07:19:17.296514+0000","last_fullsized":"2024-03-07T07:19:17.296514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417710+0000","last_change":"2024-03-07T07:19:16.307152+0000","last_active":"2024-03-07T07:19:17.417710+0000","last_peered":"2024-03-07T07:19:17.417710+0000","last_clean":"2024-03-07T07:19:17.417710+0000","last_became_active":"2024-03-07T07:19:16.306890+0000","last_became_peered":"2024-03-07T07:19:16.306890+0000","last_unstale":"2024-03-07T07:19:17.417710+0000","last_undegraded":"2024-03-07T07:19:17.417710+0000","last_fullsized":"2024-03-07T07:19:17.417710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417582+0000","last_change":"2024-03-07T07:19:15.301303+0000","last_active":"2024-03-07T07:19:17.417582+0000","last_peered":"2024-03-07T07:19:17.417582+0000","last_clean":"2024-03-07T07:19:17.417582+0000","last_became_active":"2024-03-07T07:19:15.301031+0000","last_became_peered":"2024-03-07T07:19:15.301031+0000","last_unstale":"2024-03-07T07:19:17.417582+0000","last_undegraded":"2024-03-07T07:19:17.417582+0000","last_fullsized":"2024-03-07T07:19:17.417582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870348+0000","last_change":"2024-03-07T07:19:16.305499+0000","last_active":"2024-03-07T07:19:17.870348+0000","last_peered":"2024-03-07T07:19:17.870348+0000","last_clean":"2024-03-07T07:19:17.870348+0000","last_became_active":"2024-03-07T07:19:16.304971+0000","last_became_peered":"2024-03-07T07:19:16.304971+0000","last_unstale":"2024-03-07T07:19:17.870348+0000","last_undegraded":"2024-03-07T07:19:17.870348+0000","last_fullsized":"2024-03-07T07:19:17.870348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294183+0000","last_change":"2024-03-07T07:19:15.293364+0000","last_active":"2024-03-07T07:19:17.294183+0000","last_peered":"2024-03-07T07:19:17.294183+0000","last_clean":"2024-03-07T07:19:17.294183+0000","last_became_active":"2024-03-07T07:19:15.293069+0000","last_became_peered":"2024-03-07T07:19:15.293069+0000","last_unstale":"2024-03-07T07:19:17.294183+0000","last_undegraded":"2024-03-07T07:19:17.294183+0000","last_fullsized":"2024-03-07T07:19:17.294183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297511+0000","last_change":"2024-03-07T07:19:16.304722+0000","last_active":"2024-03-07T07:19:17.297511+0000","last_peered":"2024-03-07T07:19:17.297511+0000","last_clean":"2024-03-07T07:19:17.297511+0000","last_became_active":"2024-03-07T07:19:16.304438+0000","last_became_peered":"2024-03-07T07:19:16.304438+0000","last_unstale":"2024-03-07T07:19:17.297511+0000","last_undegraded":"2024-03-07T07:19:17.297511+0000","last_fullsized":"2024-03-07T07:19:17.297511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296596+0000","last_change":"2024-03-07T07:19:15.292852+0000","last_active":"2024-03-07T07:19:17.296596+0000","last_peered":"2024-03-07T07:19:17.296596+0000","last_clean":"2024-03-07T07:19:17.296596+0000","last_became_active":"2024-03-07T07:19:15.292584+0000","last_became_peered":"2024-03-07T07:19:15.292584+0000","last_unstale":"2024-03-07T07:19:17.296596+0000","last_undegraded":"2024-03-07T07:19:17.296596+0000","last_fullsized":"2024-03-07T07:19:17.296596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068903+0000","last_change":"2024-03-07T07:19:16.303347+0000","last_active":"2024-03-07T07:19:18.068903+0000","last_peered":"2024-03-07T07:19:18.068903+0000","last_clean":"2024-03-07T07:19:18.068903+0000","last_became_active":"2024-03-07T07:19:16.303083+0000","last_became_peered":"2024-03-07T07:19:16.303083+0000","last_unstale":"2024-03-07T07:19:18.068903+0000","last_undegraded":"2024-03-07T07:19:18.068903+0000","last_fullsized":"2024-03-07T07:19:18.068903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870639+0000","last_change":"2024-03-07T07:19:15.296780+0000","last_active":"2024-03-07T07:19:17.870639+0000","last_peered":"2024-03-07T07:19:17.870639+0000","last_clean":"2024-03-07T07:19:17.870639+0000","last_became_active":"2024-03-07T07:19:15.296300+0000","last_became_peered":"2024-03-07T07:19:15.296300+0000","last_unstale":"2024-03-07T07:19:17.870639+0000","last_undegraded":"2024-03-07T07:19:17.870639+0000","last_fullsized":"2024-03-07T07:19:17.870639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870345+0000","last_change":"2024-03-07T07:19:16.297140+0000","last_active":"2024-03-07T07:19:17.870345+0000","last_peered":"2024-03-07T07:19:17.870345+0000","last_clean":"2024-03-07T07:19:17.870345+0000","last_became_active":"2024-03-07T07:19:16.296854+0000","last_became_peered":"2024-03-07T07:19:16.296854+0000","last_unstale":"2024-03-07T07:19:17.870345+0000","last_undegraded":"2024-03-07T07:19:17.870345+0000","last_fullsized":"2024-03-07T07:19:17.870345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417473+0000","last_change":"2024-03-07T07:19:15.299975+0000","last_active":"2024-03-07T07:19:17.417473+0000","last_peered":"2024-03-07T07:19:17.417473+0000","last_clean":"2024-03-07T07:19:17.417473+0000","last_became_active":"2024-03-07T07:19:15.299753+0000","last_became_peered":"2024-03-07T07:19:15.299753+0000","last_unstale":"2024-03-07T07:19:17.417473+0000","last_undegraded":"2024-03-07T07:19:17.417473+0000","last_fullsized":"2024-03-07T07:19:17.417473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068921+0000","last_change":"2024-03-07T07:19:16.307214+0000","last_active":"2024-03-07T07:19:18.068921+0000","last_peered":"2024-03-07T07:19:18.068921+0000","last_clean":"2024-03-07T07:19:18.068921+0000","last_became_active":"2024-03-07T07:19:16.306690+0000","last_became_peered":"2024-03-07T07:19:16.306690+0000","last_unstale":"2024-03-07T07:19:18.068921+0000","last_undegraded":"2024-03-07T07:19:18.068921+0000","last_fullsized":"2024-03-07T07:19:18.068921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296268+0000","last_change":"2024-03-07T07:19:15.286950+0000","last_active":"2024-03-07T07:19:17.296268+0000","last_peered":"2024-03-07T07:19:17.296268+0000","last_clean":"2024-03-07T07:19:17.296268+0000","last_became_active":"2024-03-07T07:19:15.286655+0000","last_became_peered":"2024-03-07T07:19:15.286655+0000","last_unstale":"2024-03-07T07:19:17.296268+0000","last_undegraded":"2024-03-07T07:19:17.296268+0000","last_fullsized":"2024-03-07T07:19:17.296268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296267+0000","last_change":"2024-03-07T07:19:16.299235+0000","last_active":"2024-03-07T07:19:17.296267+0000","last_peered":"2024-03-07T07:19:17.296267+0000","last_clean":"2024-03-07T07:19:17.296267+0000","last_became_active":"2024-03-07T07:19:16.298945+0000","last_became_peered":"2024-03-07T07:19:16.298945+0000","last_unstale":"2024-03-07T07:19:17.296267+0000","last_undegraded":"2024-03-07T07:19:17.296267+0000","last_fullsized":"2024-03-07T07:19:17.296267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068707+0000","last_change":"2024-03-07T07:19:15.289721+0000","last_active":"2024-03-07T07:19:18.068707+0000","last_peered":"2024-03-07T07:19:18.068707+0000","last_clean":"2024-03-07T07:19:18.068707+0000","last_became_active":"2024-03-07T07:19:15.289410+0000","last_became_peered":"2024-03-07T07:19:15.289410+0000","last_unstale":"2024-03-07T07:19:18.068707+0000","last_undegraded":"2024-03-07T07:19:18.068707+0000","last_fullsized":"2024-03-07T07:19:18.068707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068748+0000","last_change":"2024-03-07T07:19:16.306954+0000","last_active":"2024-03-07T07:19:18.068748+0000","last_peered":"2024-03-07T07:19:18.068748+0000","last_clean":"2024-03-07T07:19:18.068748+0000","last_became_active":"2024-03-07T07:19:16.306639+0000","last_became_peered":"2024-03-07T07:19:16.306639+0000","last_unstale":"2024-03-07T07:19:18.068748+0000","last_undegraded":"2024-03-07T07:19:18.068748+0000","last_fullsized":"2024-03-07T07:19:18.068748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297392+0000","last_change":"2024-03-07T07:19:15.297356+0000","last_active":"2024-03-07T07:19:17.297392+0000","last_peered":"2024-03-07T07:19:17.297392+0000","last_clean":"2024-03-07T07:19:17.297392+0000","last_became_active":"2024-03-07T07:19:15.296989+0000","last_became_peered":"2024-03-07T07:19:15.296989+0000","last_unstale":"2024-03-07T07:19:17.297392+0000","last_undegraded":"2024-03-07T07:19:17.297392+0000","last_fullsized":"2024-03-07T07:19:17.297392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:17:57.790720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069126+0000","last_change":"2024-03-07T07:19:16.306845+0000","last_active":"2024-03-07T07:19:18.069126+0000","last_peered":"2024-03-07T07:19:18.069126+0000","last_clean":"2024-03-07T07:19:18.069126+0000","last_became_active":"2024-03-07T07:19:16.306406+0000","last_became_peered":"2024-03-07T07:19:16.306406+0000","last_unstale":"2024-03-07T07:19:18.069126+0000","last_undegraded":"2024-03-07T07:19:18.069126+0000","last_fullsized":"2024-03-07T07:19:18.069126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068846+0000","last_change":"2024-03-07T07:19:15.293365+0000","last_active":"2024-03-07T07:19:18.068846+0000","last_peered":"2024-03-07T07:19:18.068846+0000","last_clean":"2024-03-07T07:19:18.068846+0000","last_became_active":"2024-03-07T07:19:15.293062+0000","last_became_peered":"2024-03-07T07:19:15.293062+0000","last_unstale":"2024-03-07T07:19:18.068846+0000","last_undegraded":"2024-03-07T07:19:18.068846+0000","last_fullsized":"2024-03-07T07:19:18.068846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069160+0000","last_change":"2024-03-07T07:19:16.307514+0000","last_active":"2024-03-07T07:19:18.069160+0000","last_peered":"2024-03-07T07:19:18.069160+0000","last_clean":"2024-03-07T07:19:18.069160+0000","last_became_active":"2024-03-07T07:19:16.307226+0000","last_became_peered":"2024-03-07T07:19:16.307226+0000","last_unstale":"2024-03-07T07:19:18.069160+0000","last_undegraded":"2024-03-07T07:19:18.069160+0000","last_fullsized":"2024-03-07T07:19:18.069160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:25:12.269915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294284+0000","last_change":"2024-03-07T07:19:15.290118+0000","last_active":"2024-03-07T07:19:17.294284+0000","last_peered":"2024-03-07T07:19:17.294284+0000","last_clean":"2024-03-07T07:19:17.294284+0000","last_became_active":"2024-03-07T07:19:15.289812+0000","last_became_peered":"2024-03-07T07:19:15.289812+0000","last_unstale":"2024-03-07T07:19:17.294284+0000","last_undegraded":"2024-03-07T07:19:17.294284+0000","last_fullsized":"2024-03-07T07:19:17.294284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417670+0000","last_change":"2024-03-07T07:19:16.303739+0000","last_active":"2024-03-07T07:19:17.417670+0000","last_peered":"2024-03-07T07:19:17.417670+0000","last_clean":"2024-03-07T07:19:17.417670+0000","last_became_active":"2024-03-07T07:19:16.303279+0000","last_became_peered":"2024-03-07T07:19:16.303279+0000","last_unstale":"2024-03-07T07:19:17.417670+0000","last_undegraded":"2024-03-07T07:19:17.417670+0000","last_fullsized":"2024-03-07T07:19:17.417670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068866+0000","last_change":"2024-03-07T07:19:15.296198+0000","last_active":"2024-03-07T07:19:18.068866+0000","last_peered":"2024-03-07T07:19:18.068866+0000","last_clean":"2024-03-07T07:19:18.068866+0000","last_became_active":"2024-03-07T07:19:15.295886+0000","last_became_peered":"2024-03-07T07:19:15.295886+0000","last_unstale":"2024-03-07T07:19:18.068866+0000","last_undegraded":"2024-03-07T07:19:18.068866+0000","last_fullsized":"2024-03-07T07:19:18.068866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069075+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:18.069075+0000","last_peered":"2024-03-07T07:19:18.069075+0000","last_clean":"2024-03-07T07:19:18.069075+0000","last_became_active":"2024-03-07T07:19:16.307119+0000","last_became_peered":"2024-03-07T07:19:16.307119+0000","last_unstale":"2024-03-07T07:19:18.069075+0000","last_undegraded":"2024-03-07T07:19:18.069075+0000","last_fullsized":"2024-03-07T07:19:18.069075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068695+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:19:18.068695+0000","last_peered":"2024-03-07T07:19:18.068695+0000","last_clean":"2024-03-07T07:19:18.068695+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:19:18.068695+0000","last_undegraded":"2024-03-07T07:19:18.068695+0000","last_fullsized":"2024-03-07T07:19:18.068695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417348+0000","last_change":"2024-03-07T07:19:15.295100+0000","last_active":"2024-03-07T07:19:17.417348+0000","last_peered":"2024-03-07T07:19:17.417348+0000","last_clean":"2024-03-07T07:19:17.417348+0000","last_became_active":"2024-03-07T07:19:15.294643+0000","last_became_peered":"2024-03-07T07:19:15.294643+0000","last_unstale":"2024-03-07T07:19:17.417348+0000","last_undegraded":"2024-03-07T07:19:17.417348+0000","last_fullsized":"2024-03-07T07:19:17.417348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.306837+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.305695+0000","last_became_peered":"2024-03-07T07:19:16.305695+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068642+0000","last_change":"2024-03-07T07:19:15.301442+0000","last_active":"2024-03-07T07:19:18.068642+0000","last_peered":"2024-03-07T07:19:18.068642+0000","last_clean":"2024-03-07T07:19:18.068642+0000","last_became_active":"2024-03-07T07:19:15.300526+0000","last_became_peered":"2024-03-07T07:19:15.300526+0000","last_unstale":"2024-03-07T07:19:18.068642+0000","last_undegraded":"2024-03-07T07:19:18.068642+0000","last_fullsized":"2024-03-07T07:19:18.068642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870083+0000","last_change":"2024-03-07T07:19:16.297399+0000","last_active":"2024-03-07T07:19:17.870083+0000","last_peered":"2024-03-07T07:19:17.870083+0000","last_clean":"2024-03-07T07:19:17.870083+0000","last_became_active":"2024-03-07T07:19:16.296941+0000","last_became_peered":"2024-03-07T07:19:16.296941+0000","last_unstale":"2024-03-07T07:19:17.870083+0000","last_undegraded":"2024-03-07T07:19:17.870083+0000","last_fullsized":"2024-03-07T07:19:17.870083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:10.125663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068824+0000","last_change":"2024-03-07T07:19:15.291261+0000","last_active":"2024-03-07T07:19:18.068824+0000","last_peered":"2024-03-07T07:19:18.068824+0000","last_clean":"2024-03-07T07:19:18.068824+0000","last_became_active":"2024-03-07T07:19:15.290977+0000","last_became_peered":"2024-03-07T07:19:15.290977+0000","last_unstale":"2024-03-07T07:19:18.068824+0000","last_undegraded":"2024-03-07T07:19:18.068824+0000","last_fullsized":"2024-03-07T07:19:18.068824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294009+0000","last_change":"2024-03-07T07:19:16.298068+0000","last_active":"2024-03-07T07:19:17.294009+0000","last_peered":"2024-03-07T07:19:17.294009+0000","last_clean":"2024-03-07T07:19:17.294009+0000","last_became_active":"2024-03-07T07:19:16.297795+0000","last_became_peered":"2024-03-07T07:19:16.297795+0000","last_unstale":"2024-03-07T07:19:17.294009+0000","last_undegraded":"2024-03-07T07:19:17.294009+0000","last_fullsized":"2024-03-07T07:19:17.294009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417588+0000","last_change":"2024-03-07T07:19:15.296795+0000","last_active":"2024-03-07T07:19:17.417588+0000","last_peered":"2024-03-07T07:19:17.417588+0000","last_clean":"2024-03-07T07:19:17.417588+0000","last_became_active":"2024-03-07T07:19:15.296534+0000","last_became_peered":"2024-03-07T07:19:15.296534+0000","last_unstale":"2024-03-07T07:19:17.417588+0000","last_undegraded":"2024-03-07T07:19:17.417588+0000","last_fullsized":"2024-03-07T07:19:17.417588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297568+0000","last_change":"2024-03-07T07:19:16.295673+0000","last_active":"2024-03-07T07:19:17.297568+0000","last_peered":"2024-03-07T07:19:17.297568+0000","last_clean":"2024-03-07T07:19:17.297568+0000","last_became_active":"2024-03-07T07:19:16.295139+0000","last_became_peered":"2024-03-07T07:19:16.295139+0000","last_unstale":"2024-03-07T07:19:17.297568+0000","last_undegraded":"2024-03-07T07:19:17.297568+0000","last_fullsized":"2024-03-07T07:19:17.297568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294148+0000","last_change":"2024-03-07T07:19:15.297274+0000","last_active":"2024-03-07T07:19:17.294148+0000","last_peered":"2024-03-07T07:19:17.294148+0000","last_clean":"2024-03-07T07:19:17.294148+0000","last_became_active":"2024-03-07T07:19:15.297000+0000","last_became_peered":"2024-03-07T07:19:15.297000+0000","last_unstale":"2024-03-07T07:19:17.294148+0000","last_undegraded":"2024-03-07T07:19:17.294148+0000","last_fullsized":"2024-03-07T07:19:17.294148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417617+0000","last_change":"2024-03-07T07:19:16.305048+0000","last_active":"2024-03-07T07:19:17.417617+0000","last_peered":"2024-03-07T07:19:17.417617+0000","last_clean":"2024-03-07T07:19:17.417617+0000","last_became_active":"2024-03-07T07:19:16.303993+0000","last_became_peered":"2024-03-07T07:19:16.303993+0000","last_unstale":"2024-03-07T07:19:17.417617+0000","last_undegraded":"2024-03-07T07:19:17.417617+0000","last_fullsized":"2024-03-07T07:19:17.417617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069397+0000","last_change":"2024-03-07T07:19:15.300909+0000","last_active":"2024-03-07T07:19:18.069397+0000","last_peered":"2024-03-07T07:19:18.069397+0000","last_clean":"2024-03-07T07:19:18.069397+0000","last_became_active":"2024-03-07T07:19:15.300573+0000","last_became_peered":"2024-03-07T07:19:15.300573+0000","last_unstale":"2024-03-07T07:19:18.069397+0000","last_undegraded":"2024-03-07T07:19:18.069397+0000","last_fullsized":"2024-03-07T07:19:18.069397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294190+0000","last_change":"2024-03-07T07:19:16.298525+0000","last_active":"2024-03-07T07:19:17.294190+0000","last_peered":"2024-03-07T07:19:17.294190+0000","last_clean":"2024-03-07T07:19:17.294190+0000","last_became_active":"2024-03-07T07:19:16.298264+0000","last_became_peered":"2024-03-07T07:19:16.298264+0000","last_unstale":"2024-03-07T07:19:17.294190+0000","last_undegraded":"2024-03-07T07:19:17.294190+0000","last_fullsized":"2024-03-07T07:19:17.294190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.295564+0000","last_change":"2024-03-07T07:19:15.291265+0000","last_active":"2024-03-07T07:19:17.295564+0000","last_peered":"2024-03-07T07:19:17.295564+0000","last_clean":"2024-03-07T07:19:17.295564+0000","last_became_active":"2024-03-07T07:19:15.290889+0000","last_became_peered":"2024-03-07T07:19:15.290889+0000","last_unstale":"2024-03-07T07:19:17.295564+0000","last_undegraded":"2024-03-07T07:19:17.295564+0000","last_fullsized":"2024-03-07T07:19:17.295564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009905+0000","last_change":"2024-03-07T07:19:16.303881+0000","last_active":"2024-03-07T07:19:18.009905+0000","last_peered":"2024-03-07T07:19:18.009905+0000","last_clean":"2024-03-07T07:19:18.009905+0000","last_became_active":"2024-03-07T07:19:16.303597+0000","last_became_peered":"2024-03-07T07:19:16.303597+0000","last_unstale":"2024-03-07T07:19:18.009905+0000","last_undegraded":"2024-03-07T07:19:18.009905+0000","last_fullsized":"2024-03-07T07:19:18.009905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296735+0000","last_change":"2024-03-07T07:19:15.296436+0000","last_active":"2024-03-07T07:19:17.296735+0000","last_peered":"2024-03-07T07:19:17.296735+0000","last_clean":"2024-03-07T07:19:17.296735+0000","last_became_active":"2024-03-07T07:19:15.296152+0000","last_became_peered":"2024-03-07T07:19:15.296152+0000","last_unstale":"2024-03-07T07:19:17.296735+0000","last_undegraded":"2024-03-07T07:19:17.296735+0000","last_fullsized":"2024-03-07T07:19:17.296735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870218+0000","last_change":"2024-03-07T07:19:16.300794+0000","last_active":"2024-03-07T07:19:17.870218+0000","last_peered":"2024-03-07T07:19:17.870218+0000","last_clean":"2024-03-07T07:19:17.870218+0000","last_became_active":"2024-03-07T07:19:16.300531+0000","last_became_peered":"2024-03-07T07:19:16.300531+0000","last_unstale":"2024-03-07T07:19:17.870218+0000","last_undegraded":"2024-03-07T07:19:17.870218+0000","last_fullsized":"2024-03-07T07:19:17.870218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068653+0000","last_change":"2024-03-07T07:19:15.301871+0000","last_active":"2024-03-07T07:19:18.068653+0000","last_peered":"2024-03-07T07:19:18.068653+0000","last_clean":"2024-03-07T07:19:18.068653+0000","last_became_active":"2024-03-07T07:19:15.301269+0000","last_became_peered":"2024-03-07T07:19:15.301269+0000","last_unstale":"2024-03-07T07:19:18.068653+0000","last_undegraded":"2024-03-07T07:19:18.068653+0000","last_fullsized":"2024-03-07T07:19:18.068653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297261+0000","last_change":"2024-03-07T07:19:16.293571+0000","last_active":"2024-03-07T07:19:17.297261+0000","last_peered":"2024-03-07T07:19:17.297261+0000","last_clean":"2024-03-07T07:19:17.297261+0000","last_became_active":"2024-03-07T07:19:16.293217+0000","last_became_peered":"2024-03-07T07:19:16.293217+0000","last_unstale":"2024-03-07T07:19:17.297261+0000","last_undegraded":"2024-03-07T07:19:17.297261+0000","last_fullsized":"2024-03-07T07:19:17.297261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:11.899192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294039+0000","last_change":"2024-03-07T07:19:15.291033+0000","last_active":"2024-03-07T07:19:17.294039+0000","last_peered":"2024-03-07T07:19:17.294039+0000","last_clean":"2024-03-07T07:19:17.294039+0000","last_became_active":"2024-03-07T07:19:15.290699+0000","last_became_peered":"2024-03-07T07:19:15.290699+0000","last_unstale":"2024-03-07T07:19:17.294039+0000","last_undegraded":"2024-03-07T07:19:17.294039+0000","last_fullsized":"2024-03-07T07:19:17.294039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009889+0000","last_change":"2024-03-07T07:19:16.300154+0000","last_active":"2024-03-07T07:19:18.009889+0000","last_peered":"2024-03-07T07:19:18.009889+0000","last_clean":"2024-03-07T07:19:18.009889+0000","last_became_active":"2024-03-07T07:19:16.299831+0000","last_became_peered":"2024-03-07T07:19:16.299831+0000","last_unstale":"2024-03-07T07:19:18.009889+0000","last_undegraded":"2024-03-07T07:19:18.009889+0000","last_fullsized":"2024-03-07T07:19:18.009889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294117+0000","last_change":"2024-03-07T07:19:15.292788+0000","last_active":"2024-03-07T07:19:17.294117+0000","last_peered":"2024-03-07T07:19:17.294117+0000","last_clean":"2024-03-07T07:19:17.294117+0000","last_became_active":"2024-03-07T07:19:15.292478+0000","last_became_peered":"2024-03-07T07:19:15.292478+0000","last_unstale":"2024-03-07T07:19:17.294117+0000","last_undegraded":"2024-03-07T07:19:17.294117+0000","last_fullsized":"2024-03-07T07:19:17.294117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.302180+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069079+0000","last_change":"2024-03-07T07:19:15.300817+0000","last_active":"2024-03-07T07:19:18.069079+0000","last_peered":"2024-03-07T07:19:18.069079+0000","last_clean":"2024-03-07T07:19:18.069079+0000","last_became_active":"2024-03-07T07:19:15.300502+0000","last_became_peered":"2024-03-07T07:19:15.300502+0000","last_unstale":"2024-03-07T07:19:18.069079+0000","last_undegraded":"2024-03-07T07:19:18.069079+0000","last_fullsized":"2024-03-07T07:19:18.069079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296526+0000","last_change":"2024-03-07T07:19:16.300613+0000","last_active":"2024-03-07T07:19:17.296526+0000","last_peered":"2024-03-07T07:19:17.296526+0000","last_clean":"2024-03-07T07:19:17.296526+0000","last_became_active":"2024-03-07T07:19:16.299969+0000","last_became_peered":"2024-03-07T07:19:16.299969+0000","last_unstale":"2024-03-07T07:19:17.296526+0000","last_undegraded":"2024-03-07T07:19:17.296526+0000","last_fullsized":"2024-03-07T07:19:17.296526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417649+0000","last_change":"2024-03-07T07:19:16.316583+0000","last_active":"2024-03-07T07:19:17.417649+0000","last_peered":"2024-03-07T07:19:17.417649+0000","last_clean":"2024-03-07T07:19:17.417649+0000","last_became_active":"2024-03-07T07:19:16.316351+0000","last_became_peered":"2024-03-07T07:19:16.316351+0000","last_unstale":"2024-03-07T07:19:17.417649+0000","last_undegraded":"2024-03-07T07:19:17.417649+0000","last_fullsized":"2024-03-07T07:19:17.417649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069099+0000","last_change":"2024-03-07T07:19:15.300879+0000","last_active":"2024-03-07T07:19:18.069099+0000","last_peered":"2024-03-07T07:19:18.069099+0000","last_clean":"2024-03-07T07:19:18.069099+0000","last_became_active":"2024-03-07T07:19:15.300594+0000","last_became_peered":"2024-03-07T07:19:15.300594+0000","last_unstale":"2024-03-07T07:19:18.069099+0000","last_undegraded":"2024-03-07T07:19:18.069099+0000","last_fullsized":"2024-03-07T07:19:18.069099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417463+0000","last_change":"2024-03-07T07:19:16.313560+0000","last_active":"2024-03-07T07:19:17.417463+0000","last_peered":"2024-03-07T07:19:17.417463+0000","last_clean":"2024-03-07T07:19:17.417463+0000","last_became_active":"2024-03-07T07:19:16.313130+0000","last_became_peered":"2024-03-07T07:19:16.313130+0000","last_unstale":"2024-03-07T07:19:17.417463+0000","last_undegraded":"2024-03-07T07:19:17.417463+0000","last_fullsized":"2024-03-07T07:19:17.417463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069186+0000","last_change":"2024-03-07T07:19:15.301563+0000","last_active":"2024-03-07T07:19:18.069186+0000","last_peered":"2024-03-07T07:19:18.069186+0000","last_clean":"2024-03-07T07:19:18.069186+0000","last_became_active":"2024-03-07T07:19:15.301282+0000","last_became_peered":"2024-03-07T07:19:15.301282+0000","last_unstale":"2024-03-07T07:19:18.069186+0000","last_undegraded":"2024-03-07T07:19:18.069186+0000","last_fullsized":"2024-03-07T07:19:18.069186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870211+0000","last_change":"2024-03-07T07:19:16.306438+0000","last_active":"2024-03-07T07:19:17.870211+0000","last_peered":"2024-03-07T07:19:17.870211+0000","last_clean":"2024-03-07T07:19:17.870211+0000","last_became_active":"2024-03-07T07:19:16.305209+0000","last_became_peered":"2024-03-07T07:19:16.305209+0000","last_unstale":"2024-03-07T07:19:17.870211+0000","last_undegraded":"2024-03-07T07:19:17.870211+0000","last_fullsized":"2024-03-07T07:19:17.870211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009925+0000","last_change":"2024-03-07T07:19:15.299866+0000","last_active":"2024-03-07T07:19:18.009925+0000","last_peered":"2024-03-07T07:19:18.009925+0000","last_clean":"2024-03-07T07:19:18.009925+0000","last_became_active":"2024-03-07T07:19:15.299621+0000","last_became_peered":"2024-03-07T07:19:15.299621+0000","last_unstale":"2024-03-07T07:19:18.009925+0000","last_undegraded":"2024-03-07T07:19:18.009925+0000","last_fullsized":"2024-03-07T07:19:18.009925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009615+0000","last_change":"2024-03-07T07:19:16.296975+0000","last_active":"2024-03-07T07:19:18.009615+0000","last_peered":"2024-03-07T07:19:18.009615+0000","last_clean":"2024-03-07T07:19:18.009615+0000","last_became_active":"2024-03-07T07:19:16.296702+0000","last_became_peered":"2024-03-07T07:19:16.296702+0000","last_unstale":"2024-03-07T07:19:18.009615+0000","last_undegraded":"2024-03-07T07:19:18.009615+0000","last_fullsized":"2024-03-07T07:19:18.009615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870446+0000","last_change":"2024-03-07T07:19:15.303925+0000","last_active":"2024-03-07T07:19:17.870446+0000","last_peered":"2024-03-07T07:19:17.870446+0000","last_clean":"2024-03-07T07:19:17.870446+0000","last_became_active":"2024-03-07T07:19:15.303646+0000","last_became_peered":"2024-03-07T07:19:15.303646+0000","last_unstale":"2024-03-07T07:19:17.870446+0000","last_undegraded":"2024-03-07T07:19:17.870446+0000","last_fullsized":"2024-03-07T07:19:17.870446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:19:20.365 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-07T07:19:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.502+0000 7fbb96887640 1 Processor -- start 2024-03-07T07:19:20.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.502+0000 7fbb96887640 1 -- start start 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.502+0000 7fbb96887640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbb900b5dd0 0x7fbb90144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.502+0000 7fbb96887640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.502+0000 7fbb96887640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb90162c80 con 0x7fbb900b43f0 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.502+0000 7fbb96887640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb90141700 con 0x7fbb900b5dd0 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.502+0000 7fbb96887640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90071dc0 con 0x7fbb90071410 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb94e24640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb8ffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbb900b5dd0 0x7fbb90144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb8f7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fbb900b43f0 0x7fbb90162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:47744/0 (socket says 172.21.15.64:47744) 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb8f7fe640 1 -- 172.21.15.64:0/4027264741 learned_addr learned my addr 172.21.15.64:0/4027264741 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb8effd640 1 -- 172.21.15.64:0/4027264741 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433103746 0 0) 0x7fbb90162c80 con 0x7fbb900b43f0 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb8effd640 1 -- 172.21.15.64:0/4027264741 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb78003520 con 0x7fbb900b43f0 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb94e24640 1 -- 172.21.15.64:0/4027264741 >> v2:172.21.15.120:3300/0 conn(0x7fbb900b5dd0 msgr2=0x7fbb90144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb94e24640 1 --2- 172.21.15.64:0/4027264741 >> v2:172.21.15.120:3300/0 conn(0x7fbb900b5dd0 0x7fbb90144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb94e24640 1 -- 172.21.15.64:0/4027264741 >> v1:172.21.15.120:6789/0 conn(0x7fbb900b43f0 legacy=0x7fbb90162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb94e24640 1 -- 172.21.15.64:0/4027264741 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900b4bd0 con 0x7fbb90071410 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.503+0000 7fbb8ffff640 1 --2- 172.21.15.64:0/4027264741 >> v2:172.21.15.120:3300/0 conn(0x7fbb900b5dd0 0x7fbb90144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb94e24640 1 --2- 172.21.15.64:0/4027264741 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90071880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009d90 tx=0x7fbb8802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62913e97d1054ba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb8effd640 1 -- 172.21.15.64:0/4027264741 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88039070 con 0x7fbb90071410 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb8effd640 1 -- 172.21.15.64:0/4027264741 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8802cbc0 con 0x7fbb90071410 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb8effd640 1 -- 172.21.15.64:0/4027264741 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88034440 con 0x7fbb90071410 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 -- 172.21.15.64:0/4027264741 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 msgr2=0x7fbb90071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 --2- 172.21.15.64:0/4027264741 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90071880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009d90 tx=0x7fbb8802be10 comp rx=0 tx=0).stop 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 -- 172.21.15.64:0/4027264741 shutdown_connections 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 --2- 172.21.15.64:0/4027264741 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90071880 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 --2- 172.21.15.64:0/4027264741 >> v2:172.21.15.120:3300/0 conn(0x7fbb900b5dd0 0x7fbb90144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 -- 172.21.15.64:0/4027264741 >> 172.21.15.64:0/4027264741 conn(0x7fbb900d0e70 msgr2=0x7fbb900d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 -- 172.21.15.64:0/4027264741 shutdown_connections 2024-03-07T07:19:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.504+0000 7fbb96887640 1 -- 172.21.15.64:0/4027264741 wait complete. 2024-03-07T07:19:20.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.505+0000 7fbb96887640 1 Processor -- start 2024-03-07T07:19:20.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb96887640 1 -- start start 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb96887640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb96887640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb900b43f0 0x7fbb9015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb96887640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb900b5dd0 0x7fbb901428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb96887640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900b5190 con 0x7fbb900b5dd0 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb96887640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90161e00 con 0x7fbb900b43f0 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb96887640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90162050 con 0x7fbb90071410 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb94e24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb900b5dd0 0x7fbb901428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.506+0000 7fbb8f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb900b43f0 0x7fbb9015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb8ffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb900b5dd0 0x7fbb901428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34192/0 (socket says 172.21.15.64:34192) 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb8f7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb900b43f0 0x7fbb9015f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:33980/0 (socket says 172.21.15.64:33980) 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 -- 172.21.15.64:0/1405249121 learned_addr learned my addr 172.21.15.64:0/1405249121 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 -- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 msgr2=0x7fbb90157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 -- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb900b43f0 msgr2=0x7fbb9015f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb900b43f0 0x7fbb9015f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 -- 172.21.15.64:0/1405249121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb901583c0 con 0x7fbb900b5dd0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb8f7fe640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb900b43f0 0x7fbb9015f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb8ffff640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb94e24640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb900b5dd0 0x7fbb901428d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009d90 tx=0x7fbb88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb8cff9640 1 -- 172.21.15.64:0/1405249121 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88039050 con 0x7fbb900b5dd0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9015e9f0 con 0x7fbb900b5dd0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb8cff9640 1 -- 172.21.15.64:0/1405249121 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88045070 con 0x7fbb900b5dd0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb8cff9640 1 -- 172.21.15.64:0/1405249121 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88009450 con 0x7fbb900b5dd0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.507+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9015a190 con 0x7fbb900b5dd0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.508+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb90062c90 con 0x7fbb900b5dd0 2024-03-07T07:19:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.509+0000 7fbb8cff9640 1 -- 172.21.15.64:0/1405249121 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbb8803e830 con 0x7fbb900b5dd0 2024-03-07T07:19:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.509+0000 7fbb8cff9640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb7c06b4d0 0x7fbb7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.509+0000 7fbb8cff9640 1 -- 172.21.15.64:0/1405249121 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb880ad030 con 0x7fbb900b5dd0 2024-03-07T07:19:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.509+0000 7fbb8ffff640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb7c06b4d0 0x7fbb7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.510+0000 7fbb8ffff640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb7c06b4d0 0x7fbb7c06d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009560 tx=0x7fbb80009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.513+0000 7fbb8cff9640 1 -- 172.21.15.64:0/1405249121 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb8807ba50 con 0x7fbb900b5dd0 2024-03-07T07:19:20.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.769+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbb900b8a10 con 0x7fbb900b5dd0 2024-03-07T07:19:20.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.769+0000 7fbb8cff9640 1 -- 172.21.15.64:0/1405249121 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1011 (secure 0 0 0) 0x7fbb8807faf0 con 0x7fbb900b5dd0 2024-03-07T07:19:20.770 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:20.770 INFO:teuthology.orchestra.run.smithi064.stdout:{"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1709795943,"num_in_osds":8,"osd_in_since":1709795934,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224247808,"bytes_avail":772869865472,"bytes_total":773094113280,"write_bytes_sec":2427,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4299}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-03-07T07:18:51.914182+0000","services":{}},"progress_events":{}} 2024-03-07T07:19:20.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb7c06b4d0 msgr2=0x7fbb7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb7c06b4d0 0x7fbb7c06d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009560 tx=0x7fbb80009190 comp rx=0 tx=0).stop 2024-03-07T07:19:20.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb900b5dd0 msgr2=0x7fbb901428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb900b5dd0 0x7fbb901428d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009d90 tx=0x7fbb88004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:20.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 shutdown_connections 2024-03-07T07:19:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb7c06b4d0 0x7fbb7c06d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb900b5dd0 0x7fbb901428d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb900b43f0 0x7fbb9015f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 --2- 172.21.15.64:0/1405249121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb90071410 0x7fbb90157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 >> 172.21.15.64:0/1405249121 conn(0x7fbb900d0e70 msgr2=0x7fbb900d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 shutdown_connections 2024-03-07T07:19:20.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.771+0000 7fbb96887640 1 -- 172.21.15.64:0/1405249121 wait complete. 2024-03-07T07:19:20.783 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-07T07:19:20.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:19:20.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 Processor -- start 2024-03-07T07:19:20.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 -- start start 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0cd8162030 0x7f0cd815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd81416d0 0x7f0cd8162410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0cd8162950 0x7f0cd80d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd80b44b0 con 0x7f0cd8162950 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd8141c60 con 0x7f0cd8162030 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.883+0000 7f0cde4dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd815ecd0 con 0x7f0cd81416d0 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cdca7a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0cd8162950 0x7f0cd80d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0cd8162030 0x7f0cd815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd77fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd81416d0 0x7f0cd8162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0cd8162030 0x7f0cd815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:33990/0 (socket says 172.21.15.64:33990) 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cdca7a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0cd8162950 0x7f0cd80d3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34200/0 (socket says 172.21.15.64:34200) 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 -- 172.21.15.64:0/2236611969 learned_addr learned my addr 172.21.15.64:0/2236611969 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 -- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd81416d0 msgr2=0x7f0cd8162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 --2- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd81416d0 0x7f0cd8162410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 -- 172.21.15.64:0/2236611969 >> v2:172.21.15.64:3300/0 conn(0x7f0cd8162950 msgr2=0x7f0cd80d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 --2- 172.21.15.64:0/2236611969 >> v2:172.21.15.64:3300/0 conn(0x7f0cd8162950 0x7f0cd80d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd7fff640 1 -- 172.21.15.64:0/2236611969 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd815efb0 con 0x7f0cd8162030 2024-03-07T07:19:20.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.884+0000 7f0cd77fe640 1 --2- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd81416d0 0x7f0cd8162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cd7fff640 1 --2- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd8162030 0x7f0cd815e680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8074d40 tx=0x7f0ccc02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aec01d83d2a5d499 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cd6ffd640 1 -- 172.21.15.64:0/2236611969 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc035020 con 0x7f0cd8162030 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cd6ffd640 1 -- 172.21.15.64:0/2236611969 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc031870 con 0x7f0cd8162030 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cd6ffd640 1 -- 172.21.15.64:0/2236611969 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc031c40 con 0x7f0cd8162030 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 -- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd8162030 msgr2=0x7f0cd815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd8162030 0x7f0cd815e680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8074d40 tx=0x7f0ccc02b910 comp rx=0 tx=0).stop 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 -- 172.21.15.64:0/2236611969 shutdown_connections 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/2236611969 >> v2:172.21.15.64:3300/0 conn(0x7f0cd8162950 0x7f0cd80d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd81416d0 0x7f0cd8162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/2236611969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd8162030 0x7f0cd815e680 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 -- 172.21.15.64:0/2236611969 >> 172.21.15.64:0/2236611969 conn(0x7f0cd8070400 msgr2=0x7f0cd80d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:20.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 -- 172.21.15.64:0/2236611969 shutdown_connections 2024-03-07T07:19:20.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.885+0000 7f0cde4dd640 1 -- 172.21.15.64:0/2236611969 wait complete. 2024-03-07T07:19:20.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.887+0000 7f0cde4dd640 1 Processor -- start 2024-03-07T07:19:20.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.887+0000 7f0cde4dd640 1 -- start start 2024-03-07T07:19:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.887+0000 7f0cde4dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd81416d0 0x7f0cd8154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.887+0000 7f0cde4dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd8162030 0x7f0cd8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.887+0000 7f0cde4dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd8162950 0x7f0cd8157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cde4dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80d39d0 con 0x7f0cd8162950 2024-03-07T07:19:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cde4dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd815f570 con 0x7f0cd81416d0 2024-03-07T07:19:20.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cde4dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80d3b50 con 0x7f0cd8162030 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd8162950 0x7f0cd8157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd8162950 0x7f0cd8157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34216/0 (socket says 172.21.15.64:34216) 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 -- 172.21.15.64:0/3348893147 learned_addr learned my addr 172.21.15.64:0/3348893147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cd7fff640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd81416d0 0x7f0cd8154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cd77fe640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd8162030 0x7f0cd8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 -- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd8162030 msgr2=0x7f0cd8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd8162030 0x7f0cd8157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 -- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd81416d0 msgr2=0x7f0cd8154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd81416d0 0x7f0cd8154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 -- 172.21.15.64:0/3348893147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd81f4420 con 0x7f0cd8162950 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cdca7a640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd8162950 0x7f0cd8157eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000efc0 tx=0x7f0cd000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cd4ff9640 1 -- 172.21.15.64:0/3348893147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0018020 con 0x7f0cd8162950 2024-03-07T07:19:20.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cd4ff9640 1 -- 172.21.15.64:0/3348893147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0014480 con 0x7f0cd8162950 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd81f4700 con 0x7f0cd8162950 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cd4ff9640 1 -- 172.21.15.64:0/3348893147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0014870 con 0x7f0cd8162950 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.888+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd81f4cc0 con 0x7f0cd8162950 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.889+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd8062c90 con 0x7f0cd8162950 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.890+0000 7f0cd4ff9640 1 -- 172.21.15.64:0/3348893147 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0cd0005d10 con 0x7f0cd8162950 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.890+0000 7f0cd4ff9640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0cc406b4d0 0x7f0cc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.890+0000 7f0cd4ff9640 1 -- 172.21.15.64:0/3348893147 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cd008af60 con 0x7f0cd8162950 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.890+0000 7f0cd7fff640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0cc406b4d0 0x7f0cc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:20.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.891+0000 7f0cd7fff640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0cc406b4d0 0x7f0cc406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc036000 tx=0x7f0ccc03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:20.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:20.894+0000 7f0cd4ff9640 1 -- 172.21.15.64:0/3348893147 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0cd0022030 con 0x7f0cd8162950 2024-03-07T07:19:21.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.072+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cd80b73b0 con 0x7f0cc406b4d0 2024-03-07T07:19:21.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.079+0000 7f0cd4ff9640 1 -- 172.21.15.64:0/3348893147 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251948 (secure 0 0 0) 0x7f0ccc085840 con 0x7f0cc406b4d0 2024-03-07T07:19:21.082 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:21.084 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:19:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0cc406b4d0 msgr2=0x7f0cc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:21.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0cc406b4d0 0x7f0cc406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc036000 tx=0x7f0ccc03c040 comp rx=0 tx=0).stop 2024-03-07T07:19:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd8162950 msgr2=0x7f0cd8157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd8162950 0x7f0cd8157eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000efc0 tx=0x7f0cd000c490 comp rx=0 tx=0).stop 2024-03-07T07:19:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 shutdown_connections 2024-03-07T07:19:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0cc406b4d0 0x7f0cc406d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:21.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0cd8162950 0x7f0cd8157eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0cd8162030 0x7f0cd8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 --2- 172.21.15.64:0/3348893147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cd81416d0 0x7f0cd8154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 >> 172.21.15.64:0/3348893147 conn(0x7f0cd8070400 msgr2=0x7f0cd80d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 shutdown_connections 2024-03-07T07:19:21.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:21.082+0000 7f0cde4dd640 1 -- 172.21.15.64:0/3348893147 wait complete. 2024-03-07T07:19:21.093 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-07T07:19:20.180659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.906365"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069031+0000","last_change":"2024-03-07T07:19:16.302696+0000","last_active":"2024-03-07T07:19:18.069031+0000","last_peered":"2024-03-07T07:19:18.069031+0000","last_clean":"2024-03-07T07:19:18.069031+0000","last_became_active":"2024-03-07T07:19:16.302389+0000","last_became_peered":"2024-03-07T07:19:16.302389+0000","last_unstale":"2024-03-07T07:19:18.069031+0000","last_undegraded":"2024-03-07T07:19:18.069031+0000","last_fullsized":"2024-03-07T07:19:18.069031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294255+0000","last_change":"2024-03-07T07:19:16.303217+0000","last_active":"2024-03-07T07:19:17.294255+0000","last_peered":"2024-03-07T07:19:17.294255+0000","last_clean":"2024-03-07T07:19:17.294255+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:17.294255+0000","last_undegraded":"2024-03-07T07:19:17.294255+0000","last_fullsized":"2024-03-07T07:19:17.294255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870246+0000","last_change":"2024-03-07T07:19:16.292858+0000","last_active":"2024-03-07T07:19:17.870246+0000","last_peered":"2024-03-07T07:19:17.870246+0000","last_clean":"2024-03-07T07:19:17.870246+0000","last_became_active":"2024-03-07T07:19:16.292593+0000","last_became_peered":"2024-03-07T07:19:16.292593+0000","last_unstale":"2024-03-07T07:19:17.870246+0000","last_undegraded":"2024-03-07T07:19:17.870246+0000","last_fullsized":"2024-03-07T07:19:17.870246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417391+0000","last_change":"2024-03-07T07:19:16.310590+0000","last_active":"2024-03-07T07:19:17.417391+0000","last_peered":"2024-03-07T07:19:17.417391+0000","last_clean":"2024-03-07T07:19:17.417391+0000","last_became_active":"2024-03-07T07:19:16.310298+0000","last_became_peered":"2024-03-07T07:19:16.310298+0000","last_unstale":"2024-03-07T07:19:17.417391+0000","last_undegraded":"2024-03-07T07:19:17.417391+0000","last_fullsized":"2024-03-07T07:19:17.417391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069132+0000","last_change":"2024-03-07T07:19:16.299586+0000","last_active":"2024-03-07T07:19:18.069132+0000","last_peered":"2024-03-07T07:19:18.069132+0000","last_clean":"2024-03-07T07:19:18.069132+0000","last_became_active":"2024-03-07T07:19:16.299265+0000","last_became_peered":"2024-03-07T07:19:16.299265+0000","last_unstale":"2024-03-07T07:19:18.069132+0000","last_undegraded":"2024-03-07T07:19:18.069132+0000","last_fullsized":"2024-03-07T07:19:18.069132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417301+0000","last_change":"2024-03-07T07:19:16.312616+0000","last_active":"2024-03-07T07:19:17.417301+0000","last_peered":"2024-03-07T07:19:17.417301+0000","last_clean":"2024-03-07T07:19:17.417301+0000","last_became_active":"2024-03-07T07:19:16.312339+0000","last_became_peered":"2024-03-07T07:19:16.312339+0000","last_unstale":"2024-03-07T07:19:17.417301+0000","last_undegraded":"2024-03-07T07:19:17.417301+0000","last_fullsized":"2024-03-07T07:19:17.417301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068749+0000","last_change":"2024-03-07T07:19:16.302351+0000","last_active":"2024-03-07T07:19:18.068749+0000","last_peered":"2024-03-07T07:19:18.068749+0000","last_clean":"2024-03-07T07:19:18.068749+0000","last_became_active":"2024-03-07T07:19:16.302071+0000","last_became_peered":"2024-03-07T07:19:16.302071+0000","last_unstale":"2024-03-07T07:19:18.068749+0000","last_undegraded":"2024-03-07T07:19:18.068749+0000","last_fullsized":"2024-03-07T07:19:18.068749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009417+0000","last_change":"2024-03-07T07:19:16.310400+0000","last_active":"2024-03-07T07:19:18.009417+0000","last_peered":"2024-03-07T07:19:18.009417+0000","last_clean":"2024-03-07T07:19:18.009417+0000","last_became_active":"2024-03-07T07:19:16.310066+0000","last_became_peered":"2024-03-07T07:19:16.310066+0000","last_unstale":"2024-03-07T07:19:18.009417+0000","last_undegraded":"2024-03-07T07:19:18.009417+0000","last_fullsized":"2024-03-07T07:19:18.009417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069013+0000","last_change":"2024-03-07T07:19:16.307888+0000","last_active":"2024-03-07T07:19:18.069013+0000","last_peered":"2024-03-07T07:19:18.069013+0000","last_clean":"2024-03-07T07:19:18.069013+0000","last_became_active":"2024-03-07T07:19:16.306871+0000","last_became_peered":"2024-03-07T07:19:16.306871+0000","last_unstale":"2024-03-07T07:19:18.069013+0000","last_undegraded":"2024-03-07T07:19:18.069013+0000","last_fullsized":"2024-03-07T07:19:18.069013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296721+0000","last_change":"2024-03-07T07:19:16.306115+0000","last_active":"2024-03-07T07:19:17.296721+0000","last_peered":"2024-03-07T07:19:17.296721+0000","last_clean":"2024-03-07T07:19:17.296721+0000","last_became_active":"2024-03-07T07:19:16.305591+0000","last_became_peered":"2024-03-07T07:19:16.305591+0000","last_unstale":"2024-03-07T07:19:17.296721+0000","last_undegraded":"2024-03-07T07:19:17.296721+0000","last_fullsized":"2024-03-07T07:19:17.296721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296408+0000","last_change":"2024-03-07T07:19:16.305713+0000","last_active":"2024-03-07T07:19:17.296408+0000","last_peered":"2024-03-07T07:19:17.296408+0000","last_clean":"2024-03-07T07:19:17.296408+0000","last_became_active":"2024-03-07T07:19:16.305408+0000","last_became_peered":"2024-03-07T07:19:16.305408+0000","last_unstale":"2024-03-07T07:19:17.296408+0000","last_undegraded":"2024-03-07T07:19:17.296408+0000","last_fullsized":"2024-03-07T07:19:17.296408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297262+0000","last_change":"2024-03-07T07:19:16.301659+0000","last_active":"2024-03-07T07:19:17.297262+0000","last_peered":"2024-03-07T07:19:17.297262+0000","last_clean":"2024-03-07T07:19:17.297262+0000","last_became_active":"2024-03-07T07:19:16.301033+0000","last_became_peered":"2024-03-07T07:19:16.301033+0000","last_unstale":"2024-03-07T07:19:17.297262+0000","last_undegraded":"2024-03-07T07:19:17.297262+0000","last_fullsized":"2024-03-07T07:19:17.297262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:34.845258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009648+0000","last_change":"2024-03-07T07:19:16.296414+0000","last_active":"2024-03-07T07:19:18.009648+0000","last_peered":"2024-03-07T07:19:18.009648+0000","last_clean":"2024-03-07T07:19:18.009648+0000","last_became_active":"2024-03-07T07:19:16.296078+0000","last_became_peered":"2024-03-07T07:19:16.296078+0000","last_unstale":"2024-03-07T07:19:18.009648+0000","last_undegraded":"2024-03-07T07:19:18.009648+0000","last_fullsized":"2024-03-07T07:19:18.009648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870192+0000","last_change":"2024-03-07T07:19:16.298815+0000","last_active":"2024-03-07T07:19:17.870192+0000","last_peered":"2024-03-07T07:19:17.870192+0000","last_clean":"2024-03-07T07:19:17.870192+0000","last_became_active":"2024-03-07T07:19:16.298560+0000","last_became_peered":"2024-03-07T07:19:16.298560+0000","last_unstale":"2024-03-07T07:19:17.870192+0000","last_undegraded":"2024-03-07T07:19:17.870192+0000","last_fullsized":"2024-03-07T07:19:17.870192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:14:50.701779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293979+0000","last_change":"2024-03-07T07:19:16.304915+0000","last_active":"2024-03-07T07:19:17.293979+0000","last_peered":"2024-03-07T07:19:17.293979+0000","last_clean":"2024-03-07T07:19:17.293979+0000","last_became_active":"2024-03-07T07:19:16.304581+0000","last_became_peered":"2024-03-07T07:19:16.304581+0000","last_unstale":"2024-03-07T07:19:17.293979+0000","last_undegraded":"2024-03-07T07:19:17.293979+0000","last_fullsized":"2024-03-07T07:19:17.293979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417242+0000","last_change":"2024-03-07T07:19:16.312660+0000","last_active":"2024-03-07T07:19:17.417242+0000","last_peered":"2024-03-07T07:19:17.417242+0000","last_clean":"2024-03-07T07:19:17.417242+0000","last_became_active":"2024-03-07T07:19:16.312356+0000","last_became_peered":"2024-03-07T07:19:16.312356+0000","last_unstale":"2024-03-07T07:19:17.417242+0000","last_undegraded":"2024-03-07T07:19:17.417242+0000","last_fullsized":"2024-03-07T07:19:17.417242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417697+0000","last_change":"2024-03-07T07:19:16.311126+0000","last_active":"2024-03-07T07:19:17.417697+0000","last_peered":"2024-03-07T07:19:17.417697+0000","last_clean":"2024-03-07T07:19:17.417697+0000","last_became_active":"2024-03-07T07:19:16.310806+0000","last_became_peered":"2024-03-07T07:19:16.310806+0000","last_unstale":"2024-03-07T07:19:17.417697+0000","last_undegraded":"2024-03-07T07:19:17.417697+0000","last_fullsized":"2024-03-07T07:19:17.417697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068899+0000","last_change":"2024-03-07T07:19:16.296921+0000","last_active":"2024-03-07T07:19:18.068899+0000","last_peered":"2024-03-07T07:19:18.068899+0000","last_clean":"2024-03-07T07:19:18.068899+0000","last_became_active":"2024-03-07T07:19:16.296657+0000","last_became_peered":"2024-03-07T07:19:16.296657+0000","last_unstale":"2024-03-07T07:19:18.068899+0000","last_undegraded":"2024-03-07T07:19:18.068899+0000","last_fullsized":"2024-03-07T07:19:18.068899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069320+0000","last_change":"2024-03-07T07:19:16.301924+0000","last_active":"2024-03-07T07:19:18.069320+0000","last_peered":"2024-03-07T07:19:18.069320+0000","last_clean":"2024-03-07T07:19:18.069320+0000","last_became_active":"2024-03-07T07:19:16.301657+0000","last_became_peered":"2024-03-07T07:19:16.301657+0000","last_unstale":"2024-03-07T07:19:18.069320+0000","last_undegraded":"2024-03-07T07:19:18.069320+0000","last_fullsized":"2024-03-07T07:19:18.069320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:16:48.547021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296490+0000","last_change":"2024-03-07T07:19:16.300281+0000","last_active":"2024-03-07T07:19:17.296490+0000","last_peered":"2024-03-07T07:19:17.296490+0000","last_clean":"2024-03-07T07:19:17.296490+0000","last_became_active":"2024-03-07T07:19:16.299775+0000","last_became_peered":"2024-03-07T07:19:16.299775+0000","last_unstale":"2024-03-07T07:19:17.296490+0000","last_undegraded":"2024-03-07T07:19:17.296490+0000","last_fullsized":"2024-03-07T07:19:17.296490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870510+0000","last_change":"2024-03-07T07:19:16.296327+0000","last_active":"2024-03-07T07:19:17.870510+0000","last_peered":"2024-03-07T07:19:17.870510+0000","last_clean":"2024-03-07T07:19:17.870510+0000","last_became_active":"2024-03-07T07:19:16.295959+0000","last_became_peered":"2024-03-07T07:19:16.295959+0000","last_unstale":"2024-03-07T07:19:17.870510+0000","last_undegraded":"2024-03-07T07:19:17.870510+0000","last_fullsized":"2024-03-07T07:19:17.870510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417371+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:17.417371+0000","last_peered":"2024-03-07T07:19:17.417371+0000","last_clean":"2024-03-07T07:19:17.417371+0000","last_became_active":"2024-03-07T07:19:16.307389+0000","last_became_peered":"2024-03-07T07:19:16.307389+0000","last_unstale":"2024-03-07T07:19:17.417371+0000","last_undegraded":"2024-03-07T07:19:17.417371+0000","last_fullsized":"2024-03-07T07:19:17.417371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009458+0000","last_change":"2024-03-07T07:19:16.295700+0000","last_active":"2024-03-07T07:19:18.009458+0000","last_peered":"2024-03-07T07:19:18.009458+0000","last_clean":"2024-03-07T07:19:18.009458+0000","last_became_active":"2024-03-07T07:19:16.295428+0000","last_became_peered":"2024-03-07T07:19:16.295428+0000","last_unstale":"2024-03-07T07:19:18.009458+0000","last_undegraded":"2024-03-07T07:19:18.009458+0000","last_fullsized":"2024-03-07T07:19:18.009458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417302+0000","last_change":"2024-03-07T07:19:16.305382+0000","last_active":"2024-03-07T07:19:17.417302+0000","last_peered":"2024-03-07T07:19:17.417302+0000","last_clean":"2024-03-07T07:19:17.417302+0000","last_became_active":"2024-03-07T07:19:16.304697+0000","last_became_peered":"2024-03-07T07:19:16.304697+0000","last_unstale":"2024-03-07T07:19:17.417302+0000","last_undegraded":"2024-03-07T07:19:17.417302+0000","last_fullsized":"2024-03-07T07:19:17.417302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009833+0000","last_change":"2024-03-07T07:19:16.295728+0000","last_active":"2024-03-07T07:19:18.009833+0000","last_peered":"2024-03-07T07:19:18.009833+0000","last_clean":"2024-03-07T07:19:18.009833+0000","last_became_active":"2024-03-07T07:19:16.295472+0000","last_became_peered":"2024-03-07T07:19:16.295472+0000","last_unstale":"2024-03-07T07:19:18.009833+0000","last_undegraded":"2024-03-07T07:19:18.009833+0000","last_fullsized":"2024-03-07T07:19:18.009833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296848+0000","last_change":"2024-03-07T07:19:16.297196+0000","last_active":"2024-03-07T07:19:17.296848+0000","last_peered":"2024-03-07T07:19:17.296848+0000","last_clean":"2024-03-07T07:19:17.296848+0000","last_became_active":"2024-03-07T07:19:16.296727+0000","last_became_peered":"2024-03-07T07:19:16.296727+0000","last_unstale":"2024-03-07T07:19:17.296848+0000","last_undegraded":"2024-03-07T07:19:17.296848+0000","last_fullsized":"2024-03-07T07:19:17.296848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294209+0000","last_change":"2024-03-07T07:19:16.299086+0000","last_active":"2024-03-07T07:19:17.294209+0000","last_peered":"2024-03-07T07:19:17.294209+0000","last_clean":"2024-03-07T07:19:17.294209+0000","last_became_active":"2024-03-07T07:19:16.298819+0000","last_became_peered":"2024-03-07T07:19:16.298819+0000","last_unstale":"2024-03-07T07:19:17.294209+0000","last_undegraded":"2024-03-07T07:19:17.294209+0000","last_fullsized":"2024-03-07T07:19:17.294209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009756+0000","last_change":"2024-03-07T07:19:16.299038+0000","last_active":"2024-03-07T07:19:18.009756+0000","last_peered":"2024-03-07T07:19:18.009756+0000","last_clean":"2024-03-07T07:19:18.009756+0000","last_became_active":"2024-03-07T07:19:16.298748+0000","last_became_peered":"2024-03-07T07:19:16.298748+0000","last_unstale":"2024-03-07T07:19:18.009756+0000","last_undegraded":"2024-03-07T07:19:18.009756+0000","last_fullsized":"2024-03-07T07:19:18.009756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870540+0000","last_change":"2024-03-07T07:19:16.293023+0000","last_active":"2024-03-07T07:19:17.870540+0000","last_peered":"2024-03-07T07:19:17.870540+0000","last_clean":"2024-03-07T07:19:17.870540+0000","last_became_active":"2024-03-07T07:19:16.292670+0000","last_became_peered":"2024-03-07T07:19:16.292670+0000","last_unstale":"2024-03-07T07:19:17.870540+0000","last_undegraded":"2024-03-07T07:19:17.870540+0000","last_fullsized":"2024-03-07T07:19:17.870540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417541+0000","last_change":"2024-03-07T07:19:16.310502+0000","last_active":"2024-03-07T07:19:17.417541+0000","last_peered":"2024-03-07T07:19:17.417541+0000","last_clean":"2024-03-07T07:19:17.417541+0000","last_became_active":"2024-03-07T07:19:16.309883+0000","last_became_peered":"2024-03-07T07:19:16.309883+0000","last_unstale":"2024-03-07T07:19:17.417541+0000","last_undegraded":"2024-03-07T07:19:17.417541+0000","last_fullsized":"2024-03-07T07:19:17.417541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068772+0000","last_change":"2024-03-07T07:19:16.306042+0000","last_active":"2024-03-07T07:19:18.068772+0000","last_peered":"2024-03-07T07:19:18.068772+0000","last_clean":"2024-03-07T07:19:18.068772+0000","last_became_active":"2024-03-07T07:19:16.304954+0000","last_became_peered":"2024-03-07T07:19:16.304954+0000","last_unstale":"2024-03-07T07:19:18.068772+0000","last_undegraded":"2024-03-07T07:19:18.068772+0000","last_fullsized":"2024-03-07T07:19:18.068772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296496+0000","last_change":"2024-03-07T07:19:16.300717+0000","last_active":"2024-03-07T07:19:17.296496+0000","last_peered":"2024-03-07T07:19:17.296496+0000","last_clean":"2024-03-07T07:19:17.296496+0000","last_became_active":"2024-03-07T07:19:16.300170+0000","last_became_peered":"2024-03-07T07:19:16.300170+0000","last_unstale":"2024-03-07T07:19:17.296496+0000","last_undegraded":"2024-03-07T07:19:17.296496+0000","last_fullsized":"2024-03-07T07:19:17.296496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297170+0000","last_change":"2024-03-07T07:19:16.299268+0000","last_active":"2024-03-07T07:19:17.297170+0000","last_peered":"2024-03-07T07:19:17.297170+0000","last_clean":"2024-03-07T07:19:17.297170+0000","last_became_active":"2024-03-07T07:19:16.298987+0000","last_became_peered":"2024-03-07T07:19:16.298987+0000","last_unstale":"2024-03-07T07:19:17.297170+0000","last_undegraded":"2024-03-07T07:19:17.297170+0000","last_fullsized":"2024-03-07T07:19:17.297170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:19.940197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009780+0000","last_change":"2024-03-07T07:19:15.298629+0000","last_active":"2024-03-07T07:19:18.009780+0000","last_peered":"2024-03-07T07:19:18.009780+0000","last_clean":"2024-03-07T07:19:18.009780+0000","last_became_active":"2024-03-07T07:19:15.298338+0000","last_became_peered":"2024-03-07T07:19:15.298338+0000","last_unstale":"2024-03-07T07:19:18.009780+0000","last_undegraded":"2024-03-07T07:19:18.009780+0000","last_fullsized":"2024-03-07T07:19:18.009780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870658+0000","last_change":"2024-03-07T07:19:16.299764+0000","last_active":"2024-03-07T07:19:17.870658+0000","last_peered":"2024-03-07T07:19:17.870658+0000","last_clean":"2024-03-07T07:19:17.870658+0000","last_became_active":"2024-03-07T07:19:16.299493+0000","last_became_peered":"2024-03-07T07:19:16.299493+0000","last_unstale":"2024-03-07T07:19:17.870658+0000","last_undegraded":"2024-03-07T07:19:17.870658+0000","last_fullsized":"2024-03-07T07:19:17.870658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296626+0000","last_change":"2024-03-07T07:19:15.285743+0000","last_active":"2024-03-07T07:19:17.296626+0000","last_peered":"2024-03-07T07:19:17.296626+0000","last_clean":"2024-03-07T07:19:17.296626+0000","last_became_active":"2024-03-07T07:19:15.285484+0000","last_became_peered":"2024-03-07T07:19:15.285484+0000","last_unstale":"2024-03-07T07:19:17.296626+0000","last_undegraded":"2024-03-07T07:19:17.296626+0000","last_fullsized":"2024-03-07T07:19:17.296626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009544+0000","last_change":"2024-03-07T07:19:15.294301+0000","last_active":"2024-03-07T07:19:18.009544+0000","last_peered":"2024-03-07T07:19:18.009544+0000","last_clean":"2024-03-07T07:19:18.009544+0000","last_became_active":"2024-03-07T07:19:15.294023+0000","last_became_peered":"2024-03-07T07:19:15.294023+0000","last_unstale":"2024-03-07T07:19:18.009544+0000","last_undegraded":"2024-03-07T07:19:18.009544+0000","last_fullsized":"2024-03-07T07:19:18.009544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009524+0000","last_change":"2024-03-07T07:19:16.299031+0000","last_active":"2024-03-07T07:19:18.009524+0000","last_peered":"2024-03-07T07:19:18.009524+0000","last_clean":"2024-03-07T07:19:18.009524+0000","last_became_active":"2024-03-07T07:19:16.298760+0000","last_became_peered":"2024-03-07T07:19:16.298760+0000","last_unstale":"2024-03-07T07:19:18.009524+0000","last_undegraded":"2024-03-07T07:19:18.009524+0000","last_fullsized":"2024-03-07T07:19:18.009524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417339+0000","last_change":"2024-03-07T07:19:15.292620+0000","last_active":"2024-03-07T07:19:17.417339+0000","last_peered":"2024-03-07T07:19:17.417339+0000","last_clean":"2024-03-07T07:19:17.417339+0000","last_became_active":"2024-03-07T07:19:15.292337+0000","last_became_peered":"2024-03-07T07:19:15.292337+0000","last_unstale":"2024-03-07T07:19:17.417339+0000","last_undegraded":"2024-03-07T07:19:17.417339+0000","last_fullsized":"2024-03-07T07:19:17.417339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293989+0000","last_change":"2024-03-07T07:19:16.297970+0000","last_active":"2024-03-07T07:19:17.293989+0000","last_peered":"2024-03-07T07:19:17.293989+0000","last_clean":"2024-03-07T07:19:17.293989+0000","last_became_active":"2024-03-07T07:19:16.297661+0000","last_became_peered":"2024-03-07T07:19:16.297661+0000","last_unstale":"2024-03-07T07:19:17.293989+0000","last_undegraded":"2024-03-07T07:19:17.293989+0000","last_fullsized":"2024-03-07T07:19:17.293989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296514+0000","last_change":"2024-03-07T07:19:15.295817+0000","last_active":"2024-03-07T07:19:17.296514+0000","last_peered":"2024-03-07T07:19:17.296514+0000","last_clean":"2024-03-07T07:19:17.296514+0000","last_became_active":"2024-03-07T07:19:15.295538+0000","last_became_peered":"2024-03-07T07:19:15.295538+0000","last_unstale":"2024-03-07T07:19:17.296514+0000","last_undegraded":"2024-03-07T07:19:17.296514+0000","last_fullsized":"2024-03-07T07:19:17.296514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417710+0000","last_change":"2024-03-07T07:19:16.307152+0000","last_active":"2024-03-07T07:19:17.417710+0000","last_peered":"2024-03-07T07:19:17.417710+0000","last_clean":"2024-03-07T07:19:17.417710+0000","last_became_active":"2024-03-07T07:19:16.306890+0000","last_became_peered":"2024-03-07T07:19:16.306890+0000","last_unstale":"2024-03-07T07:19:17.417710+0000","last_undegraded":"2024-03-07T07:19:17.417710+0000","last_fullsized":"2024-03-07T07:19:17.417710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417582+0000","last_change":"2024-03-07T07:19:15.301303+0000","last_active":"2024-03-07T07:19:17.417582+0000","last_peered":"2024-03-07T07:19:17.417582+0000","last_clean":"2024-03-07T07:19:17.417582+0000","last_became_active":"2024-03-07T07:19:15.301031+0000","last_became_peered":"2024-03-07T07:19:15.301031+0000","last_unstale":"2024-03-07T07:19:17.417582+0000","last_undegraded":"2024-03-07T07:19:17.417582+0000","last_fullsized":"2024-03-07T07:19:17.417582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870348+0000","last_change":"2024-03-07T07:19:16.305499+0000","last_active":"2024-03-07T07:19:17.870348+0000","last_peered":"2024-03-07T07:19:17.870348+0000","last_clean":"2024-03-07T07:19:17.870348+0000","last_became_active":"2024-03-07T07:19:16.304971+0000","last_became_peered":"2024-03-07T07:19:16.304971+0000","last_unstale":"2024-03-07T07:19:17.870348+0000","last_undegraded":"2024-03-07T07:19:17.870348+0000","last_fullsized":"2024-03-07T07:19:17.870348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294183+0000","last_change":"2024-03-07T07:19:15.293364+0000","last_active":"2024-03-07T07:19:17.294183+0000","last_peered":"2024-03-07T07:19:17.294183+0000","last_clean":"2024-03-07T07:19:17.294183+0000","last_became_active":"2024-03-07T07:19:15.293069+0000","last_became_peered":"2024-03-07T07:19:15.293069+0000","last_unstale":"2024-03-07T07:19:17.294183+0000","last_undegraded":"2024-03-07T07:19:17.294183+0000","last_fullsized":"2024-03-07T07:19:17.294183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297511+0000","last_change":"2024-03-07T07:19:16.304722+0000","last_active":"2024-03-07T07:19:17.297511+0000","last_peered":"2024-03-07T07:19:17.297511+0000","last_clean":"2024-03-07T07:19:17.297511+0000","last_became_active":"2024-03-07T07:19:16.304438+0000","last_became_peered":"2024-03-07T07:19:16.304438+0000","last_unstale":"2024-03-07T07:19:17.297511+0000","last_undegraded":"2024-03-07T07:19:17.297511+0000","last_fullsized":"2024-03-07T07:19:17.297511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296596+0000","last_change":"2024-03-07T07:19:15.292852+0000","last_active":"2024-03-07T07:19:17.296596+0000","last_peered":"2024-03-07T07:19:17.296596+0000","last_clean":"2024-03-07T07:19:17.296596+0000","last_became_active":"2024-03-07T07:19:15.292584+0000","last_became_peered":"2024-03-07T07:19:15.292584+0000","last_unstale":"2024-03-07T07:19:17.296596+0000","last_undegraded":"2024-03-07T07:19:17.296596+0000","last_fullsized":"2024-03-07T07:19:17.296596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068903+0000","last_change":"2024-03-07T07:19:16.303347+0000","last_active":"2024-03-07T07:19:18.068903+0000","last_peered":"2024-03-07T07:19:18.068903+0000","last_clean":"2024-03-07T07:19:18.068903+0000","last_became_active":"2024-03-07T07:19:16.303083+0000","last_became_peered":"2024-03-07T07:19:16.303083+0000","last_unstale":"2024-03-07T07:19:18.068903+0000","last_undegraded":"2024-03-07T07:19:18.068903+0000","last_fullsized":"2024-03-07T07:19:18.068903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870639+0000","last_change":"2024-03-07T07:19:15.296780+0000","last_active":"2024-03-07T07:19:17.870639+0000","last_peered":"2024-03-07T07:19:17.870639+0000","last_clean":"2024-03-07T07:19:17.870639+0000","last_became_active":"2024-03-07T07:19:15.296300+0000","last_became_peered":"2024-03-07T07:19:15.296300+0000","last_unstale":"2024-03-07T07:19:17.870639+0000","last_undegraded":"2024-03-07T07:19:17.870639+0000","last_fullsized":"2024-03-07T07:19:17.870639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870345+0000","last_change":"2024-03-07T07:19:16.297140+0000","last_active":"2024-03-07T07:19:17.870345+0000","last_peered":"2024-03-07T07:19:17.870345+0000","last_clean":"2024-03-07T07:19:17.870345+0000","last_became_active":"2024-03-07T07:19:16.296854+0000","last_became_peered":"2024-03-07T07:19:16.296854+0000","last_unstale":"2024-03-07T07:19:17.870345+0000","last_undegraded":"2024-03-07T07:19:17.870345+0000","last_fullsized":"2024-03-07T07:19:17.870345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417473+0000","last_change":"2024-03-07T07:19:15.299975+0000","last_active":"2024-03-07T07:19:17.417473+0000","last_peered":"2024-03-07T07:19:17.417473+0000","last_clean":"2024-03-07T07:19:17.417473+0000","last_became_active":"2024-03-07T07:19:15.299753+0000","last_became_peered":"2024-03-07T07:19:15.299753+0000","last_unstale":"2024-03-07T07:19:17.417473+0000","last_undegraded":"2024-03-07T07:19:17.417473+0000","last_fullsized":"2024-03-07T07:19:17.417473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068921+0000","last_change":"2024-03-07T07:19:16.307214+0000","last_active":"2024-03-07T07:19:18.068921+0000","last_peered":"2024-03-07T07:19:18.068921+0000","last_clean":"2024-03-07T07:19:18.068921+0000","last_became_active":"2024-03-07T07:19:16.306690+0000","last_became_peered":"2024-03-07T07:19:16.306690+0000","last_unstale":"2024-03-07T07:19:18.068921+0000","last_undegraded":"2024-03-07T07:19:18.068921+0000","last_fullsized":"2024-03-07T07:19:18.068921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296268+0000","last_change":"2024-03-07T07:19:15.286950+0000","last_active":"2024-03-07T07:19:17.296268+0000","last_peered":"2024-03-07T07:19:17.296268+0000","last_clean":"2024-03-07T07:19:17.296268+0000","last_became_active":"2024-03-07T07:19:15.286655+0000","last_became_peered":"2024-03-07T07:19:15.286655+0000","last_unstale":"2024-03-07T07:19:17.296268+0000","last_undegraded":"2024-03-07T07:19:17.296268+0000","last_fullsized":"2024-03-07T07:19:17.296268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296267+0000","last_change":"2024-03-07T07:19:16.299235+0000","last_active":"2024-03-07T07:19:17.296267+0000","last_peered":"2024-03-07T07:19:17.296267+0000","last_clean":"2024-03-07T07:19:17.296267+0000","last_became_active":"2024-03-07T07:19:16.298945+0000","last_became_peered":"2024-03-07T07:19:16.298945+0000","last_unstale":"2024-03-07T07:19:17.296267+0000","last_undegraded":"2024-03-07T07:19:17.296267+0000","last_fullsized":"2024-03-07T07:19:17.296267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068707+0000","last_change":"2024-03-07T07:19:15.289721+0000","last_active":"2024-03-07T07:19:18.068707+0000","last_peered":"2024-03-07T07:19:18.068707+0000","last_clean":"2024-03-07T07:19:18.068707+0000","last_became_active":"2024-03-07T07:19:15.289410+0000","last_became_peered":"2024-03-07T07:19:15.289410+0000","last_unstale":"2024-03-07T07:19:18.068707+0000","last_undegraded":"2024-03-07T07:19:18.068707+0000","last_fullsized":"2024-03-07T07:19:18.068707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068748+0000","last_change":"2024-03-07T07:19:16.306954+0000","last_active":"2024-03-07T07:19:18.068748+0000","last_peered":"2024-03-07T07:19:18.068748+0000","last_clean":"2024-03-07T07:19:18.068748+0000","last_became_active":"2024-03-07T07:19:16.306639+0000","last_became_peered":"2024-03-07T07:19:16.306639+0000","last_unstale":"2024-03-07T07:19:18.068748+0000","last_undegraded":"2024-03-07T07:19:18.068748+0000","last_fullsized":"2024-03-07T07:19:18.068748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297392+0000","last_change":"2024-03-07T07:19:15.297356+0000","last_active":"2024-03-07T07:19:17.297392+0000","last_peered":"2024-03-07T07:19:17.297392+0000","last_clean":"2024-03-07T07:19:17.297392+0000","last_became_active":"2024-03-07T07:19:15.296989+0000","last_became_peered":"2024-03-07T07:19:15.296989+0000","last_unstale":"2024-03-07T07:19:17.297392+0000","last_undegraded":"2024-03-07T07:19:17.297392+0000","last_fullsized":"2024-03-07T07:19:17.297392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:17:57.790720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069126+0000","last_change":"2024-03-07T07:19:16.306845+0000","last_active":"2024-03-07T07:19:18.069126+0000","last_peered":"2024-03-07T07:19:18.069126+0000","last_clean":"2024-03-07T07:19:18.069126+0000","last_became_active":"2024-03-07T07:19:16.306406+0000","last_became_peered":"2024-03-07T07:19:16.306406+0000","last_unstale":"2024-03-07T07:19:18.069126+0000","last_undegraded":"2024-03-07T07:19:18.069126+0000","last_fullsized":"2024-03-07T07:19:18.069126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068846+0000","last_change":"2024-03-07T07:19:15.293365+0000","last_active":"2024-03-07T07:19:18.068846+0000","last_peered":"2024-03-07T07:19:18.068846+0000","last_clean":"2024-03-07T07:19:18.068846+0000","last_became_active":"2024-03-07T07:19:15.293062+0000","last_became_peered":"2024-03-07T07:19:15.293062+0000","last_unstale":"2024-03-07T07:19:18.068846+0000","last_undegraded":"2024-03-07T07:19:18.068846+0000","last_fullsized":"2024-03-07T07:19:18.068846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069160+0000","last_change":"2024-03-07T07:19:16.307514+0000","last_active":"2024-03-07T07:19:18.069160+0000","last_peered":"2024-03-07T07:19:18.069160+0000","last_clean":"2024-03-07T07:19:18.069160+0000","last_became_active":"2024-03-07T07:19:16.307226+0000","last_became_peered":"2024-03-07T07:19:16.307226+0000","last_unstale":"2024-03-07T07:19:18.069160+0000","last_undegraded":"2024-03-07T07:19:18.069160+0000","last_fullsized":"2024-03-07T07:19:18.069160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:25:12.269915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294284+0000","last_change":"2024-03-07T07:19:15.290118+0000","last_active":"2024-03-07T07:19:17.294284+0000","last_peered":"2024-03-07T07:19:17.294284+0000","last_clean":"2024-03-07T07:19:17.294284+0000","last_became_active":"2024-03-07T07:19:15.289812+0000","last_became_peered":"2024-03-07T07:19:15.289812+0000","last_unstale":"2024-03-07T07:19:17.294284+0000","last_undegraded":"2024-03-07T07:19:17.294284+0000","last_fullsized":"2024-03-07T07:19:17.294284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417670+0000","last_change":"2024-03-07T07:19:16.303739+0000","last_active":"2024-03-07T07:19:17.417670+0000","last_peered":"2024-03-07T07:19:17.417670+0000","last_clean":"2024-03-07T07:19:17.417670+0000","last_became_active":"2024-03-07T07:19:16.303279+0000","last_became_peered":"2024-03-07T07:19:16.303279+0000","last_unstale":"2024-03-07T07:19:17.417670+0000","last_undegraded":"2024-03-07T07:19:17.417670+0000","last_fullsized":"2024-03-07T07:19:17.417670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068866+0000","last_change":"2024-03-07T07:19:15.296198+0000","last_active":"2024-03-07T07:19:18.068866+0000","last_peered":"2024-03-07T07:19:18.068866+0000","last_clean":"2024-03-07T07:19:18.068866+0000","last_became_active":"2024-03-07T07:19:15.295886+0000","last_became_peered":"2024-03-07T07:19:15.295886+0000","last_unstale":"2024-03-07T07:19:18.068866+0000","last_undegraded":"2024-03-07T07:19:18.068866+0000","last_fullsized":"2024-03-07T07:19:18.068866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069075+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:18.069075+0000","last_peered":"2024-03-07T07:19:18.069075+0000","last_clean":"2024-03-07T07:19:18.069075+0000","last_became_active":"2024-03-07T07:19:16.307119+0000","last_became_peered":"2024-03-07T07:19:16.307119+0000","last_unstale":"2024-03-07T07:19:18.069075+0000","last_undegraded":"2024-03-07T07:19:18.069075+0000","last_fullsized":"2024-03-07T07:19:18.069075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068695+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:19:18.068695+0000","last_peered":"2024-03-07T07:19:18.068695+0000","last_clean":"2024-03-07T07:19:18.068695+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:19:18.068695+0000","last_undegraded":"2024-03-07T07:19:18.068695+0000","last_fullsized":"2024-03-07T07:19:18.068695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417348+0000","last_change":"2024-03-07T07:19:15.295100+0000","last_active":"2024-03-07T07:19:17.417348+0000","last_peered":"2024-03-07T07:19:17.417348+0000","last_clean":"2024-03-07T07:19:17.417348+0000","last_became_active":"2024-03-07T07:19:15.294643+0000","last_became_peered":"2024-03-07T07:19:15.294643+0000","last_unstale":"2024-03-07T07:19:17.417348+0000","last_undegraded":"2024-03-07T07:19:17.417348+0000","last_fullsized":"2024-03-07T07:19:17.417348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.306837+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.305695+0000","last_became_peered":"2024-03-07T07:19:16.305695+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068642+0000","last_change":"2024-03-07T07:19:15.301442+0000","last_active":"2024-03-07T07:19:18.068642+0000","last_peered":"2024-03-07T07:19:18.068642+0000","last_clean":"2024-03-07T07:19:18.068642+0000","last_became_active":"2024-03-07T07:19:15.300526+0000","last_became_peered":"2024-03-07T07:19:15.300526+0000","last_unstale":"2024-03-07T07:19:18.068642+0000","last_undegraded":"2024-03-07T07:19:18.068642+0000","last_fullsized":"2024-03-07T07:19:18.068642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870083+0000","last_change":"2024-03-07T07:19:16.297399+0000","last_active":"2024-03-07T07:19:17.870083+0000","last_peered":"2024-03-07T07:19:17.870083+0000","last_clean":"2024-03-07T07:19:17.870083+0000","last_became_active":"2024-03-07T07:19:16.296941+0000","last_became_peered":"2024-03-07T07:19:16.296941+0000","last_unstale":"2024-03-07T07:19:17.870083+0000","last_undegraded":"2024-03-07T07:19:17.870083+0000","last_fullsized":"2024-03-07T07:19:17.870083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:10.125663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068824+0000","last_change":"2024-03-07T07:19:15.291261+0000","last_active":"2024-03-07T07:19:18.068824+0000","last_peered":"2024-03-07T07:19:18.068824+0000","last_clean":"2024-03-07T07:19:18.068824+0000","last_became_active":"2024-03-07T07:19:15.290977+0000","last_became_peered":"2024-03-07T07:19:15.290977+0000","last_unstale":"2024-03-07T07:19:18.068824+0000","last_undegraded":"2024-03-07T07:19:18.068824+0000","last_fullsized":"2024-03-07T07:19:18.068824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294009+0000","last_change":"2024-03-07T07:19:16.298068+0000","last_active":"2024-03-07T07:19:17.294009+0000","last_peered":"2024-03-07T07:19:17.294009+0000","last_clean":"2024-03-07T07:19:17.294009+0000","last_became_active":"2024-03-07T07:19:16.297795+0000","last_became_peered":"2024-03-07T07:19:16.297795+0000","last_unstale":"2024-03-07T07:19:17.294009+0000","last_undegraded":"2024-03-07T07:19:17.294009+0000","last_fullsized":"2024-03-07T07:19:17.294009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417588+0000","last_change":"2024-03-07T07:19:15.296795+0000","last_active":"2024-03-07T07:19:17.417588+0000","last_peered":"2024-03-07T07:19:17.417588+0000","last_clean":"2024-03-07T07:19:17.417588+0000","last_became_active":"2024-03-07T07:19:15.296534+0000","last_became_peered":"2024-03-07T07:19:15.296534+0000","last_unstale":"2024-03-07T07:19:17.417588+0000","last_undegraded":"2024-03-07T07:19:17.417588+0000","last_fullsized":"2024-03-07T07:19:17.417588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297568+0000","last_change":"2024-03-07T07:19:16.295673+0000","last_active":"2024-03-07T07:19:17.297568+0000","last_peered":"2024-03-07T07:19:17.297568+0000","last_clean":"2024-03-07T07:19:17.297568+0000","last_became_active":"2024-03-07T07:19:16.295139+0000","last_became_peered":"2024-03-07T07:19:16.295139+0000","last_unstale":"2024-03-07T07:19:17.297568+0000","last_undegraded":"2024-03-07T07:19:17.297568+0000","last_fullsized":"2024-03-07T07:19:17.297568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294148+0000","last_change":"2024-03-07T07:19:15.297274+0000","last_active":"2024-03-07T07:19:17.294148+0000","last_peered":"2024-03-07T07:19:17.294148+0000","last_clean":"2024-03-07T07:19:17.294148+0000","last_became_active":"2024-03-07T07:19:15.297000+0000","last_became_peered":"2024-03-07T07:19:15.297000+0000","last_unstale":"2024-03-07T07:19:17.294148+0000","last_undegraded":"2024-03-07T07:19:17.294148+0000","last_fullsized":"2024-03-07T07:19:17.294148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417617+0000","last_change":"2024-03-07T07:19:16.305048+0000","last_active":"2024-03-07T07:19:17.417617+0000","last_peered":"2024-03-07T07:19:17.417617+0000","last_clean":"2024-03-07T07:19:17.417617+0000","last_became_active":"2024-03-07T07:19:16.303993+0000","last_became_peered":"2024-03-07T07:19:16.303993+0000","last_unstale":"2024-03-07T07:19:17.417617+0000","last_undegraded":"2024-03-07T07:19:17.417617+0000","last_fullsized":"2024-03-07T07:19:17.417617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069397+0000","last_change":"2024-03-07T07:19:15.300909+0000","last_active":"2024-03-07T07:19:18.069397+0000","last_peered":"2024-03-07T07:19:18.069397+0000","last_clean":"2024-03-07T07:19:18.069397+0000","last_became_active":"2024-03-07T07:19:15.300573+0000","last_became_peered":"2024-03-07T07:19:15.300573+0000","last_unstale":"2024-03-07T07:19:18.069397+0000","last_undegraded":"2024-03-07T07:19:18.069397+0000","last_fullsized":"2024-03-07T07:19:18.069397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294190+0000","last_change":"2024-03-07T07:19:16.298525+0000","last_active":"2024-03-07T07:19:17.294190+0000","last_peered":"2024-03-07T07:19:17.294190+0000","last_clean":"2024-03-07T07:19:17.294190+0000","last_became_active":"2024-03-07T07:19:16.298264+0000","last_became_peered":"2024-03-07T07:19:16.298264+0000","last_unstale":"2024-03-07T07:19:17.294190+0000","last_undegraded":"2024-03-07T07:19:17.294190+0000","last_fullsized":"2024-03-07T07:19:17.294190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.295564+0000","last_change":"2024-03-07T07:19:15.291265+0000","last_active":"2024-03-07T07:19:17.295564+0000","last_peered":"2024-03-07T07:19:17.295564+0000","last_clean":"2024-03-07T07:19:17.295564+0000","last_became_active":"2024-03-07T07:19:15.290889+0000","last_became_peered":"2024-03-07T07:19:15.290889+0000","last_unstale":"2024-03-07T07:19:17.295564+0000","last_undegraded":"2024-03-07T07:19:17.295564+0000","last_fullsized":"2024-03-07T07:19:17.295564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009905+0000","last_change":"2024-03-07T07:19:16.303881+0000","last_active":"2024-03-07T07:19:18.009905+0000","last_peered":"2024-03-07T07:19:18.009905+0000","last_clean":"2024-03-07T07:19:18.009905+0000","last_became_active":"2024-03-07T07:19:16.303597+0000","last_became_peered":"2024-03-07T07:19:16.303597+0000","last_unstale":"2024-03-07T07:19:18.009905+0000","last_undegraded":"2024-03-07T07:19:18.009905+0000","last_fullsized":"2024-03-07T07:19:18.009905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296735+0000","last_change":"2024-03-07T07:19:15.296436+0000","last_active":"2024-03-07T07:19:17.296735+0000","last_peered":"2024-03-07T07:19:17.296735+0000","last_clean":"2024-03-07T07:19:17.296735+0000","last_became_active":"2024-03-07T07:19:15.296152+0000","last_became_peered":"2024-03-07T07:19:15.296152+0000","last_unstale":"2024-03-07T07:19:17.296735+0000","last_undegraded":"2024-03-07T07:19:17.296735+0000","last_fullsized":"2024-03-07T07:19:17.296735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870218+0000","last_change":"2024-03-07T07:19:16.300794+0000","last_active":"2024-03-07T07:19:17.870218+0000","last_peered":"2024-03-07T07:19:17.870218+0000","last_clean":"2024-03-07T07:19:17.870218+0000","last_became_active":"2024-03-07T07:19:16.300531+0000","last_became_peered":"2024-03-07T07:19:16.300531+0000","last_unstale":"2024-03-07T07:19:17.870218+0000","last_undegraded":"2024-03-07T07:19:17.870218+0000","last_fullsized":"2024-03-07T07:19:17.870218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068653+0000","last_change":"2024-03-07T07:19:15.301871+0000","last_active":"2024-03-07T07:19:18.068653+0000","last_peered":"2024-03-07T07:19:18.068653+0000","last_clean":"2024-03-07T07:19:18.068653+0000","last_became_active":"2024-03-07T07:19:15.301269+0000","last_became_peered":"2024-03-07T07:19:15.301269+0000","last_unstale":"2024-03-07T07:19:18.068653+0000","last_undegraded":"2024-03-07T07:19:18.068653+0000","last_fullsized":"2024-03-07T07:19:18.068653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297261+0000","last_change":"2024-03-07T07:19:16.293571+0000","last_active":"2024-03-07T07:19:17.297261+0000","last_peered":"2024-03-07T07:19:17.297261+0000","last_clean":"2024-03-07T07:19:17.297261+0000","last_became_active":"2024-03-07T07:19:16.293217+0000","last_became_peered":"2024-03-07T07:19:16.293217+0000","last_unstale":"2024-03-07T07:19:17.297261+0000","last_undegraded":"2024-03-07T07:19:17.297261+0000","last_fullsized":"2024-03-07T07:19:17.297261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:11.899192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294039+0000","last_change":"2024-03-07T07:19:15.291033+0000","last_active":"2024-03-07T07:19:17.294039+0000","last_peered":"2024-03-07T07:19:17.294039+0000","last_clean":"2024-03-07T07:19:17.294039+0000","last_became_active":"2024-03-07T07:19:15.290699+0000","last_became_peered":"2024-03-07T07:19:15.290699+0000","last_unstale":"2024-03-07T07:19:17.294039+0000","last_undegraded":"2024-03-07T07:19:17.294039+0000","last_fullsized":"2024-03-07T07:19:17.294039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009889+0000","last_change":"2024-03-07T07:19:16.300154+0000","last_active":"2024-03-07T07:19:18.009889+0000","last_peered":"2024-03-07T07:19:18.009889+0000","last_clean":"2024-03-07T07:19:18.009889+0000","last_became_active":"2024-03-07T07:19:16.299831+0000","last_became_peered":"2024-03-07T07:19:16.299831+0000","last_unstale":"2024-03-07T07:19:18.009889+0000","last_undegraded":"2024-03-07T07:19:18.009889+0000","last_fullsized":"2024-03-07T07:19:18.009889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294117+0000","last_change":"2024-03-07T07:19:15.292788+0000","last_active":"2024-03-07T07:19:17.294117+0000","last_peered":"2024-03-07T07:19:17.294117+0000","last_clean":"2024-03-07T07:19:17.294117+0000","last_became_active":"2024-03-07T07:19:15.292478+0000","last_became_peered":"2024-03-07T07:19:15.292478+0000","last_unstale":"2024-03-07T07:19:17.294117+0000","last_undegraded":"2024-03-07T07:19:17.294117+0000","last_fullsized":"2024-03-07T07:19:17.294117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.302180+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069079+0000","last_change":"2024-03-07T07:19:15.300817+0000","last_active":"2024-03-07T07:19:18.069079+0000","last_peered":"2024-03-07T07:19:18.069079+0000","last_clean":"2024-03-07T07:19:18.069079+0000","last_became_active":"2024-03-07T07:19:15.300502+0000","last_became_peered":"2024-03-07T07:19:15.300502+0000","last_unstale":"2024-03-07T07:19:18.069079+0000","last_undegraded":"2024-03-07T07:19:18.069079+0000","last_fullsized":"2024-03-07T07:19:18.069079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296526+0000","last_change":"2024-03-07T07:19:16.300613+0000","last_active":"2024-03-07T07:19:17.296526+0000","last_peered":"2024-03-07T07:19:17.296526+0000","last_clean":"2024-03-07T07:19:17.296526+0000","last_became_active":"2024-03-07T07:19:16.299969+0000","last_became_peered":"2024-03-07T07:19:16.299969+0000","last_unstale":"2024-03-07T07:19:17.296526+0000","last_undegraded":"2024-03-07T07:19:17.296526+0000","last_fullsized":"2024-03-07T07:19:17.296526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417649+0000","last_change":"2024-03-07T07:19:16.316583+0000","last_active":"2024-03-07T07:19:17.417649+0000","last_peered":"2024-03-07T07:19:17.417649+0000","last_clean":"2024-03-07T07:19:17.417649+0000","last_became_active":"2024-03-07T07:19:16.316351+0000","last_became_peered":"2024-03-07T07:19:16.316351+0000","last_unstale":"2024-03-07T07:19:17.417649+0000","last_undegraded":"2024-03-07T07:19:17.417649+0000","last_fullsized":"2024-03-07T07:19:17.417649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069099+0000","last_change":"2024-03-07T07:19:15.300879+0000","last_active":"2024-03-07T07:19:18.069099+0000","last_peered":"2024-03-07T07:19:18.069099+0000","last_clean":"2024-03-07T07:19:18.069099+0000","last_became_active":"2024-03-07T07:19:15.300594+0000","last_became_peered":"2024-03-07T07:19:15.300594+0000","last_unstale":"2024-03-07T07:19:18.069099+0000","last_undegraded":"2024-03-07T07:19:18.069099+0000","last_fullsized":"2024-03-07T07:19:18.069099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417463+0000","last_change":"2024-03-07T07:19:16.313560+0000","last_active":"2024-03-07T07:19:17.417463+0000","last_peered":"2024-03-07T07:19:17.417463+0000","last_clean":"2024-03-07T07:19:17.417463+0000","last_became_active":"2024-03-07T07:19:16.313130+0000","last_became_peered":"2024-03-07T07:19:16.313130+0000","last_unstale":"2024-03-07T07:19:17.417463+0000","last_undegraded":"2024-03-07T07:19:17.417463+0000","last_fullsized":"2024-03-07T07:19:17.417463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069186+0000","last_change":"2024-03-07T07:19:15.301563+0000","last_active":"2024-03-07T07:19:18.069186+0000","last_peered":"2024-03-07T07:19:18.069186+0000","last_clean":"2024-03-07T07:19:18.069186+0000","last_became_active":"2024-03-07T07:19:15.301282+0000","last_became_peered":"2024-03-07T07:19:15.301282+0000","last_unstale":"2024-03-07T07:19:18.069186+0000","last_undegraded":"2024-03-07T07:19:18.069186+0000","last_fullsized":"2024-03-07T07:19:18.069186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870211+0000","last_change":"2024-03-07T07:19:16.306438+0000","last_active":"2024-03-07T07:19:17.870211+0000","last_peered":"2024-03-07T07:19:17.870211+0000","last_clean":"2024-03-07T07:19:17.870211+0000","last_became_active":"2024-03-07T07:19:16.305209+0000","last_became_peered":"2024-03-07T07:19:16.305209+0000","last_unstale":"2024-03-07T07:19:17.870211+0000","last_undegraded":"2024-03-07T07:19:17.870211+0000","last_fullsized":"2024-03-07T07:19:17.870211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009925+0000","last_change":"2024-03-07T07:19:15.299866+0000","last_active":"2024-03-07T07:19:18.009925+0000","last_peered":"2024-03-07T07:19:18.009925+0000","last_clean":"2024-03-07T07:19:18.009925+0000","last_became_active":"2024-03-07T07:19:15.299621+0000","last_became_peered":"2024-03-07T07:19:15.299621+0000","last_unstale":"2024-03-07T07:19:18.009925+0000","last_undegraded":"2024-03-07T07:19:18.009925+0000","last_fullsized":"2024-03-07T07:19:18.009925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009615+0000","last_change":"2024-03-07T07:19:16.296975+0000","last_active":"2024-03-07T07:19:18.009615+0000","last_peered":"2024-03-07T07:19:18.009615+0000","last_clean":"2024-03-07T07:19:18.009615+0000","last_became_active":"2024-03-07T07:19:16.296702+0000","last_became_peered":"2024-03-07T07:19:16.296702+0000","last_unstale":"2024-03-07T07:19:18.009615+0000","last_undegraded":"2024-03-07T07:19:18.009615+0000","last_fullsized":"2024-03-07T07:19:18.009615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870446+0000","last_change":"2024-03-07T07:19:15.303925+0000","last_active":"2024-03-07T07:19:17.870446+0000","last_peered":"2024-03-07T07:19:17.870446+0000","last_clean":"2024-03-07T07:19:17.870446+0000","last_became_active":"2024-03-07T07:19:15.303646+0000","last_became_peered":"2024-03-07T07:19:15.303646+0000","last_unstale":"2024-03-07T07:19:17.870446+0000","last_undegraded":"2024-03-07T07:19:17.870446+0000","last_fullsized":"2024-03-07T07:19:17.870446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:19:24.099 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:19:24.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.213+0000 7f394f9c9640 1 Processor -- start 2024-03-07T07:19:24.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.213+0000 7f394f9c9640 1 -- start start 2024-03-07T07:19:24.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394f9c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f39480b5dd0 0x7f3948144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394f9c9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f3948162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394f9c9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39480b4bd0 con 0x7f3948162ab0 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394f9c9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3948141700 con 0x7f39480b5dd0 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394f9c9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3948161e00 con 0x7f39480b43f0 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394cf64640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f3948162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394d765640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f39480b5dd0 0x7f3948144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394df66640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f3948162ab0 0x7f3948071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:47752/0 (socket says 172.21.15.64:47752) 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394cf64640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f3948162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:56762/0 (socket says 172.21.15.64:56762) 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.214+0000 7f394df66640 1 -- 172.21.15.64:0/2224805524 learned_addr learned my addr 172.21.15.64:0/2224805524 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.215+0000 7f394cf64640 1 -- 172.21.15.64:0/2224805524 >> v2:172.21.15.120:3300/0 conn(0x7f39480b5dd0 msgr2=0x7f3948144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.215+0000 7f394cf64640 1 --2- 172.21.15.64:0/2224805524 >> v2:172.21.15.120:3300/0 conn(0x7f39480b5dd0 0x7f3948144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.215+0000 7f394cf64640 1 -- 172.21.15.64:0/2224805524 >> v1:172.21.15.120:6789/0 conn(0x7f3948162ab0 legacy=0x7f3948071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.215+0000 7f394cf64640 1 -- 172.21.15.64:0/2224805524 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39480b4eb0 con 0x7f39480b43f0 2024-03-07T07:19:24.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.215+0000 7f394d765640 1 --2- 172.21.15.64:0/2224805524 >> v2:172.21.15.120:3300/0 conn(0x7f39480b5dd0 0x7f3948144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.215+0000 7f394cf64640 1 --2- 172.21.15.64:0/2224805524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f3948162570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3948161f80 tx=0x7f393c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12814bdc0331d63a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f3947fff640 1 -- 172.21.15.64:0/2224805524 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c031770 con 0x7f39480b43f0 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f3947fff640 1 -- 172.21.15.64:0/2224805524 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c037070 con 0x7f39480b43f0 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f3947fff640 1 -- 172.21.15.64:0/2224805524 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c031b30 con 0x7f39480b43f0 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 -- 172.21.15.64:0/2224805524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 msgr2=0x7f3948162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2224805524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f3948162570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3948161f80 tx=0x7f393c02b810 comp rx=0 tx=0).stop 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 -- 172.21.15.64:0/2224805524 shutdown_connections 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2224805524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f3948162570 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2224805524 >> v2:172.21.15.120:3300/0 conn(0x7f39480b5dd0 0x7f3948144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 -- 172.21.15.64:0/2224805524 >> 172.21.15.64:0/2224805524 conn(0x7f39480d0e70 msgr2=0x7f39480d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 -- 172.21.15.64:0/2224805524 shutdown_connections 2024-03-07T07:19:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.216+0000 7f394f9c9640 1 -- 172.21.15.64:0/2224805524 wait complete. 2024-03-07T07:19:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.217+0000 7f394f9c9640 1 Processor -- start 2024-03-07T07:19:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.217+0000 7f394f9c9640 1 -- start start 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394f9c9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f394813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394f9c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39480b5dd0 0x7f394813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394f9c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39481f4f60 0x7f39481f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394f9c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39480b61b0 con 0x7f39481f4f60 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394f9c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948162050 con 0x7f39480b5dd0 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394f9c9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39480b4870 con 0x7f39480b43f0 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394df66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39481f4f60 0x7f39481f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394df66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39481f4f60 0x7f39481f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34222/0 (socket says 172.21.15.64:34222) 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394cf64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39480b5dd0 0x7f394813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394df66640 1 -- 172.21.15.64:0/2731811985 learned_addr learned my addr 172.21.15.64:0/2731811985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.218+0000 7f394d765640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f394813f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394cf64640 1 -- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 msgr2=0x7f394813f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394cf64640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f394813f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394cf64640 1 -- 172.21.15.64:0/2731811985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39481f4f60 msgr2=0x7f39481f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394cf64640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39481f4f60 0x7f39481f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394cf64640 1 -- 172.21.15.64:0/2731811985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39481f4d40 con 0x7f39480b5dd0 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394d765640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f394813f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394cf64640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39480b5dd0 0x7f394813fc00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f393c033000 tx=0x7f393c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f3945ffb640 1 -- 172.21.15.64:0/2731811985 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c045600 con 0x7f39480b5dd0 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f3945ffb640 1 -- 172.21.15.64:0/2731811985 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c0457a0 con 0x7f39480b5dd0 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f3945ffb640 1 -- 172.21.15.64:0/2731811985 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c045b70 con 0x7f39480b5dd0 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39481f4a60 con 0x7f39480b5dd0 2024-03-07T07:19:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.219+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3948159b40 con 0x7f39480b5dd0 2024-03-07T07:19:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.220+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3948062c90 con 0x7f39480b5dd0 2024-03-07T07:19:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.221+0000 7f3945ffb640 1 -- 172.21.15.64:0/2731811985 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f393c045d10 con 0x7f39480b5dd0 2024-03-07T07:19:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.221+0000 7f3945ffb640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f393406b4d0 0x7f393406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.221+0000 7f3945ffb640 1 -- 172.21.15.64:0/2731811985 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f393c0abce0 con 0x7f39480b5dd0 2024-03-07T07:19:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.221+0000 7f394d765640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f393406b4d0 0x7f393406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.222+0000 7f394d765640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f393406b4d0 0x7f393406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3940009560 tx=0x7f3940009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.226+0000 7f3945ffb640 1 -- 172.21.15.64:0/2731811985 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f393c040070 con 0x7f39480b5dd0 2024-03-07T07:19:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.406+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39481619b0 con 0x7f393406b4d0 2024-03-07T07:19:24.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.413+0000 7f3945ffb640 1 -- 172.21.15.64:0/2731811985 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251948 (secure 0 0 0) 0x7f39481619b0 con 0x7f393406b4d0 2024-03-07T07:19:24.415 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:24.416 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f393406b4d0 msgr2=0x7f393406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f393406b4d0 0x7f393406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3940009560 tx=0x7f3940009190 comp rx=0 tx=0).stop 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39480b5dd0 msgr2=0x7f394813fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39480b5dd0 0x7f394813fc00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f393c033000 tx=0x7f393c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 shutdown_connections 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f393406b4d0 0x7f393406d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39481f4f60 0x7f39481f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f39480b5dd0 0x7f394813fc00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 --2- 172.21.15.64:0/2731811985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f39480b43f0 0x7f394813f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.415+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 >> 172.21.15.64:0/2731811985 conn(0x7f39480d0e70 msgr2=0x7f39480d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:24.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.416+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 shutdown_connections 2024-03-07T07:19:24.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.416+0000 7f394f9c9640 1 -- 172.21.15.64:0/2731811985 wait complete. 2024-03-07T07:19:24.426 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-07T07:19:24.181242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001326"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069031+0000","last_change":"2024-03-07T07:19:16.302696+0000","last_active":"2024-03-07T07:19:18.069031+0000","last_peered":"2024-03-07T07:19:18.069031+0000","last_clean":"2024-03-07T07:19:18.069031+0000","last_became_active":"2024-03-07T07:19:16.302389+0000","last_became_peered":"2024-03-07T07:19:16.302389+0000","last_unstale":"2024-03-07T07:19:18.069031+0000","last_undegraded":"2024-03-07T07:19:18.069031+0000","last_fullsized":"2024-03-07T07:19:18.069031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294255+0000","last_change":"2024-03-07T07:19:16.303217+0000","last_active":"2024-03-07T07:19:17.294255+0000","last_peered":"2024-03-07T07:19:17.294255+0000","last_clean":"2024-03-07T07:19:17.294255+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:17.294255+0000","last_undegraded":"2024-03-07T07:19:17.294255+0000","last_fullsized":"2024-03-07T07:19:17.294255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870246+0000","last_change":"2024-03-07T07:19:16.292858+0000","last_active":"2024-03-07T07:19:17.870246+0000","last_peered":"2024-03-07T07:19:17.870246+0000","last_clean":"2024-03-07T07:19:17.870246+0000","last_became_active":"2024-03-07T07:19:16.292593+0000","last_became_peered":"2024-03-07T07:19:16.292593+0000","last_unstale":"2024-03-07T07:19:17.870246+0000","last_undegraded":"2024-03-07T07:19:17.870246+0000","last_fullsized":"2024-03-07T07:19:17.870246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417391+0000","last_change":"2024-03-07T07:19:16.310590+0000","last_active":"2024-03-07T07:19:17.417391+0000","last_peered":"2024-03-07T07:19:17.417391+0000","last_clean":"2024-03-07T07:19:17.417391+0000","last_became_active":"2024-03-07T07:19:16.310298+0000","last_became_peered":"2024-03-07T07:19:16.310298+0000","last_unstale":"2024-03-07T07:19:17.417391+0000","last_undegraded":"2024-03-07T07:19:17.417391+0000","last_fullsized":"2024-03-07T07:19:17.417391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069132+0000","last_change":"2024-03-07T07:19:16.299586+0000","last_active":"2024-03-07T07:19:18.069132+0000","last_peered":"2024-03-07T07:19:18.069132+0000","last_clean":"2024-03-07T07:19:18.069132+0000","last_became_active":"2024-03-07T07:19:16.299265+0000","last_became_peered":"2024-03-07T07:19:16.299265+0000","last_unstale":"2024-03-07T07:19:18.069132+0000","last_undegraded":"2024-03-07T07:19:18.069132+0000","last_fullsized":"2024-03-07T07:19:18.069132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417301+0000","last_change":"2024-03-07T07:19:16.312616+0000","last_active":"2024-03-07T07:19:17.417301+0000","last_peered":"2024-03-07T07:19:17.417301+0000","last_clean":"2024-03-07T07:19:17.417301+0000","last_became_active":"2024-03-07T07:19:16.312339+0000","last_became_peered":"2024-03-07T07:19:16.312339+0000","last_unstale":"2024-03-07T07:19:17.417301+0000","last_undegraded":"2024-03-07T07:19:17.417301+0000","last_fullsized":"2024-03-07T07:19:17.417301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068749+0000","last_change":"2024-03-07T07:19:16.302351+0000","last_active":"2024-03-07T07:19:18.068749+0000","last_peered":"2024-03-07T07:19:18.068749+0000","last_clean":"2024-03-07T07:19:18.068749+0000","last_became_active":"2024-03-07T07:19:16.302071+0000","last_became_peered":"2024-03-07T07:19:16.302071+0000","last_unstale":"2024-03-07T07:19:18.068749+0000","last_undegraded":"2024-03-07T07:19:18.068749+0000","last_fullsized":"2024-03-07T07:19:18.068749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009417+0000","last_change":"2024-03-07T07:19:16.310400+0000","last_active":"2024-03-07T07:19:18.009417+0000","last_peered":"2024-03-07T07:19:18.009417+0000","last_clean":"2024-03-07T07:19:18.009417+0000","last_became_active":"2024-03-07T07:19:16.310066+0000","last_became_peered":"2024-03-07T07:19:16.310066+0000","last_unstale":"2024-03-07T07:19:18.009417+0000","last_undegraded":"2024-03-07T07:19:18.009417+0000","last_fullsized":"2024-03-07T07:19:18.009417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069013+0000","last_change":"2024-03-07T07:19:16.307888+0000","last_active":"2024-03-07T07:19:18.069013+0000","last_peered":"2024-03-07T07:19:18.069013+0000","last_clean":"2024-03-07T07:19:18.069013+0000","last_became_active":"2024-03-07T07:19:16.306871+0000","last_became_peered":"2024-03-07T07:19:16.306871+0000","last_unstale":"2024-03-07T07:19:18.069013+0000","last_undegraded":"2024-03-07T07:19:18.069013+0000","last_fullsized":"2024-03-07T07:19:18.069013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296721+0000","last_change":"2024-03-07T07:19:16.306115+0000","last_active":"2024-03-07T07:19:17.296721+0000","last_peered":"2024-03-07T07:19:17.296721+0000","last_clean":"2024-03-07T07:19:17.296721+0000","last_became_active":"2024-03-07T07:19:16.305591+0000","last_became_peered":"2024-03-07T07:19:16.305591+0000","last_unstale":"2024-03-07T07:19:17.296721+0000","last_undegraded":"2024-03-07T07:19:17.296721+0000","last_fullsized":"2024-03-07T07:19:17.296721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296408+0000","last_change":"2024-03-07T07:19:16.305713+0000","last_active":"2024-03-07T07:19:17.296408+0000","last_peered":"2024-03-07T07:19:17.296408+0000","last_clean":"2024-03-07T07:19:17.296408+0000","last_became_active":"2024-03-07T07:19:16.305408+0000","last_became_peered":"2024-03-07T07:19:16.305408+0000","last_unstale":"2024-03-07T07:19:17.296408+0000","last_undegraded":"2024-03-07T07:19:17.296408+0000","last_fullsized":"2024-03-07T07:19:17.296408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297262+0000","last_change":"2024-03-07T07:19:16.301659+0000","last_active":"2024-03-07T07:19:17.297262+0000","last_peered":"2024-03-07T07:19:17.297262+0000","last_clean":"2024-03-07T07:19:17.297262+0000","last_became_active":"2024-03-07T07:19:16.301033+0000","last_became_peered":"2024-03-07T07:19:16.301033+0000","last_unstale":"2024-03-07T07:19:17.297262+0000","last_undegraded":"2024-03-07T07:19:17.297262+0000","last_fullsized":"2024-03-07T07:19:17.297262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:34.845258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009648+0000","last_change":"2024-03-07T07:19:16.296414+0000","last_active":"2024-03-07T07:19:18.009648+0000","last_peered":"2024-03-07T07:19:18.009648+0000","last_clean":"2024-03-07T07:19:18.009648+0000","last_became_active":"2024-03-07T07:19:16.296078+0000","last_became_peered":"2024-03-07T07:19:16.296078+0000","last_unstale":"2024-03-07T07:19:18.009648+0000","last_undegraded":"2024-03-07T07:19:18.009648+0000","last_fullsized":"2024-03-07T07:19:18.009648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870192+0000","last_change":"2024-03-07T07:19:16.298815+0000","last_active":"2024-03-07T07:19:17.870192+0000","last_peered":"2024-03-07T07:19:17.870192+0000","last_clean":"2024-03-07T07:19:17.870192+0000","last_became_active":"2024-03-07T07:19:16.298560+0000","last_became_peered":"2024-03-07T07:19:16.298560+0000","last_unstale":"2024-03-07T07:19:17.870192+0000","last_undegraded":"2024-03-07T07:19:17.870192+0000","last_fullsized":"2024-03-07T07:19:17.870192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:14:50.701779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293979+0000","last_change":"2024-03-07T07:19:16.304915+0000","last_active":"2024-03-07T07:19:17.293979+0000","last_peered":"2024-03-07T07:19:17.293979+0000","last_clean":"2024-03-07T07:19:17.293979+0000","last_became_active":"2024-03-07T07:19:16.304581+0000","last_became_peered":"2024-03-07T07:19:16.304581+0000","last_unstale":"2024-03-07T07:19:17.293979+0000","last_undegraded":"2024-03-07T07:19:17.293979+0000","last_fullsized":"2024-03-07T07:19:17.293979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417242+0000","last_change":"2024-03-07T07:19:16.312660+0000","last_active":"2024-03-07T07:19:17.417242+0000","last_peered":"2024-03-07T07:19:17.417242+0000","last_clean":"2024-03-07T07:19:17.417242+0000","last_became_active":"2024-03-07T07:19:16.312356+0000","last_became_peered":"2024-03-07T07:19:16.312356+0000","last_unstale":"2024-03-07T07:19:17.417242+0000","last_undegraded":"2024-03-07T07:19:17.417242+0000","last_fullsized":"2024-03-07T07:19:17.417242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417697+0000","last_change":"2024-03-07T07:19:16.311126+0000","last_active":"2024-03-07T07:19:17.417697+0000","last_peered":"2024-03-07T07:19:17.417697+0000","last_clean":"2024-03-07T07:19:17.417697+0000","last_became_active":"2024-03-07T07:19:16.310806+0000","last_became_peered":"2024-03-07T07:19:16.310806+0000","last_unstale":"2024-03-07T07:19:17.417697+0000","last_undegraded":"2024-03-07T07:19:17.417697+0000","last_fullsized":"2024-03-07T07:19:17.417697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068899+0000","last_change":"2024-03-07T07:19:16.296921+0000","last_active":"2024-03-07T07:19:18.068899+0000","last_peered":"2024-03-07T07:19:18.068899+0000","last_clean":"2024-03-07T07:19:18.068899+0000","last_became_active":"2024-03-07T07:19:16.296657+0000","last_became_peered":"2024-03-07T07:19:16.296657+0000","last_unstale":"2024-03-07T07:19:18.068899+0000","last_undegraded":"2024-03-07T07:19:18.068899+0000","last_fullsized":"2024-03-07T07:19:18.068899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069320+0000","last_change":"2024-03-07T07:19:16.301924+0000","last_active":"2024-03-07T07:19:18.069320+0000","last_peered":"2024-03-07T07:19:18.069320+0000","last_clean":"2024-03-07T07:19:18.069320+0000","last_became_active":"2024-03-07T07:19:16.301657+0000","last_became_peered":"2024-03-07T07:19:16.301657+0000","last_unstale":"2024-03-07T07:19:18.069320+0000","last_undegraded":"2024-03-07T07:19:18.069320+0000","last_fullsized":"2024-03-07T07:19:18.069320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:16:48.547021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296490+0000","last_change":"2024-03-07T07:19:16.300281+0000","last_active":"2024-03-07T07:19:17.296490+0000","last_peered":"2024-03-07T07:19:17.296490+0000","last_clean":"2024-03-07T07:19:17.296490+0000","last_became_active":"2024-03-07T07:19:16.299775+0000","last_became_peered":"2024-03-07T07:19:16.299775+0000","last_unstale":"2024-03-07T07:19:17.296490+0000","last_undegraded":"2024-03-07T07:19:17.296490+0000","last_fullsized":"2024-03-07T07:19:17.296490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870510+0000","last_change":"2024-03-07T07:19:16.296327+0000","last_active":"2024-03-07T07:19:17.870510+0000","last_peered":"2024-03-07T07:19:17.870510+0000","last_clean":"2024-03-07T07:19:17.870510+0000","last_became_active":"2024-03-07T07:19:16.295959+0000","last_became_peered":"2024-03-07T07:19:16.295959+0000","last_unstale":"2024-03-07T07:19:17.870510+0000","last_undegraded":"2024-03-07T07:19:17.870510+0000","last_fullsized":"2024-03-07T07:19:17.870510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417371+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:17.417371+0000","last_peered":"2024-03-07T07:19:17.417371+0000","last_clean":"2024-03-07T07:19:17.417371+0000","last_became_active":"2024-03-07T07:19:16.307389+0000","last_became_peered":"2024-03-07T07:19:16.307389+0000","last_unstale":"2024-03-07T07:19:17.417371+0000","last_undegraded":"2024-03-07T07:19:17.417371+0000","last_fullsized":"2024-03-07T07:19:17.417371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009458+0000","last_change":"2024-03-07T07:19:16.295700+0000","last_active":"2024-03-07T07:19:18.009458+0000","last_peered":"2024-03-07T07:19:18.009458+0000","last_clean":"2024-03-07T07:19:18.009458+0000","last_became_active":"2024-03-07T07:19:16.295428+0000","last_became_peered":"2024-03-07T07:19:16.295428+0000","last_unstale":"2024-03-07T07:19:18.009458+0000","last_undegraded":"2024-03-07T07:19:18.009458+0000","last_fullsized":"2024-03-07T07:19:18.009458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417302+0000","last_change":"2024-03-07T07:19:16.305382+0000","last_active":"2024-03-07T07:19:17.417302+0000","last_peered":"2024-03-07T07:19:17.417302+0000","last_clean":"2024-03-07T07:19:17.417302+0000","last_became_active":"2024-03-07T07:19:16.304697+0000","last_became_peered":"2024-03-07T07:19:16.304697+0000","last_unstale":"2024-03-07T07:19:17.417302+0000","last_undegraded":"2024-03-07T07:19:17.417302+0000","last_fullsized":"2024-03-07T07:19:17.417302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009833+0000","last_change":"2024-03-07T07:19:16.295728+0000","last_active":"2024-03-07T07:19:18.009833+0000","last_peered":"2024-03-07T07:19:18.009833+0000","last_clean":"2024-03-07T07:19:18.009833+0000","last_became_active":"2024-03-07T07:19:16.295472+0000","last_became_peered":"2024-03-07T07:19:16.295472+0000","last_unstale":"2024-03-07T07:19:18.009833+0000","last_undegraded":"2024-03-07T07:19:18.009833+0000","last_fullsized":"2024-03-07T07:19:18.009833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296848+0000","last_change":"2024-03-07T07:19:16.297196+0000","last_active":"2024-03-07T07:19:17.296848+0000","last_peered":"2024-03-07T07:19:17.296848+0000","last_clean":"2024-03-07T07:19:17.296848+0000","last_became_active":"2024-03-07T07:19:16.296727+0000","last_became_peered":"2024-03-07T07:19:16.296727+0000","last_unstale":"2024-03-07T07:19:17.296848+0000","last_undegraded":"2024-03-07T07:19:17.296848+0000","last_fullsized":"2024-03-07T07:19:17.296848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294209+0000","last_change":"2024-03-07T07:19:16.299086+0000","last_active":"2024-03-07T07:19:17.294209+0000","last_peered":"2024-03-07T07:19:17.294209+0000","last_clean":"2024-03-07T07:19:17.294209+0000","last_became_active":"2024-03-07T07:19:16.298819+0000","last_became_peered":"2024-03-07T07:19:16.298819+0000","last_unstale":"2024-03-07T07:19:17.294209+0000","last_undegraded":"2024-03-07T07:19:17.294209+0000","last_fullsized":"2024-03-07T07:19:17.294209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009756+0000","last_change":"2024-03-07T07:19:16.299038+0000","last_active":"2024-03-07T07:19:18.009756+0000","last_peered":"2024-03-07T07:19:18.009756+0000","last_clean":"2024-03-07T07:19:18.009756+0000","last_became_active":"2024-03-07T07:19:16.298748+0000","last_became_peered":"2024-03-07T07:19:16.298748+0000","last_unstale":"2024-03-07T07:19:18.009756+0000","last_undegraded":"2024-03-07T07:19:18.009756+0000","last_fullsized":"2024-03-07T07:19:18.009756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870540+0000","last_change":"2024-03-07T07:19:16.293023+0000","last_active":"2024-03-07T07:19:17.870540+0000","last_peered":"2024-03-07T07:19:17.870540+0000","last_clean":"2024-03-07T07:19:17.870540+0000","last_became_active":"2024-03-07T07:19:16.292670+0000","last_became_peered":"2024-03-07T07:19:16.292670+0000","last_unstale":"2024-03-07T07:19:17.870540+0000","last_undegraded":"2024-03-07T07:19:17.870540+0000","last_fullsized":"2024-03-07T07:19:17.870540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417541+0000","last_change":"2024-03-07T07:19:16.310502+0000","last_active":"2024-03-07T07:19:17.417541+0000","last_peered":"2024-03-07T07:19:17.417541+0000","last_clean":"2024-03-07T07:19:17.417541+0000","last_became_active":"2024-03-07T07:19:16.309883+0000","last_became_peered":"2024-03-07T07:19:16.309883+0000","last_unstale":"2024-03-07T07:19:17.417541+0000","last_undegraded":"2024-03-07T07:19:17.417541+0000","last_fullsized":"2024-03-07T07:19:17.417541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068772+0000","last_change":"2024-03-07T07:19:16.306042+0000","last_active":"2024-03-07T07:19:18.068772+0000","last_peered":"2024-03-07T07:19:18.068772+0000","last_clean":"2024-03-07T07:19:18.068772+0000","last_became_active":"2024-03-07T07:19:16.304954+0000","last_became_peered":"2024-03-07T07:19:16.304954+0000","last_unstale":"2024-03-07T07:19:18.068772+0000","last_undegraded":"2024-03-07T07:19:18.068772+0000","last_fullsized":"2024-03-07T07:19:18.068772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296496+0000","last_change":"2024-03-07T07:19:16.300717+0000","last_active":"2024-03-07T07:19:17.296496+0000","last_peered":"2024-03-07T07:19:17.296496+0000","last_clean":"2024-03-07T07:19:17.296496+0000","last_became_active":"2024-03-07T07:19:16.300170+0000","last_became_peered":"2024-03-07T07:19:16.300170+0000","last_unstale":"2024-03-07T07:19:17.296496+0000","last_undegraded":"2024-03-07T07:19:17.296496+0000","last_fullsized":"2024-03-07T07:19:17.296496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297170+0000","last_change":"2024-03-07T07:19:16.299268+0000","last_active":"2024-03-07T07:19:17.297170+0000","last_peered":"2024-03-07T07:19:17.297170+0000","last_clean":"2024-03-07T07:19:17.297170+0000","last_became_active":"2024-03-07T07:19:16.298987+0000","last_became_peered":"2024-03-07T07:19:16.298987+0000","last_unstale":"2024-03-07T07:19:17.297170+0000","last_undegraded":"2024-03-07T07:19:17.297170+0000","last_fullsized":"2024-03-07T07:19:17.297170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:19.940197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009780+0000","last_change":"2024-03-07T07:19:15.298629+0000","last_active":"2024-03-07T07:19:18.009780+0000","last_peered":"2024-03-07T07:19:18.009780+0000","last_clean":"2024-03-07T07:19:18.009780+0000","last_became_active":"2024-03-07T07:19:15.298338+0000","last_became_peered":"2024-03-07T07:19:15.298338+0000","last_unstale":"2024-03-07T07:19:18.009780+0000","last_undegraded":"2024-03-07T07:19:18.009780+0000","last_fullsized":"2024-03-07T07:19:18.009780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870658+0000","last_change":"2024-03-07T07:19:16.299764+0000","last_active":"2024-03-07T07:19:17.870658+0000","last_peered":"2024-03-07T07:19:17.870658+0000","last_clean":"2024-03-07T07:19:17.870658+0000","last_became_active":"2024-03-07T07:19:16.299493+0000","last_became_peered":"2024-03-07T07:19:16.299493+0000","last_unstale":"2024-03-07T07:19:17.870658+0000","last_undegraded":"2024-03-07T07:19:17.870658+0000","last_fullsized":"2024-03-07T07:19:17.870658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296626+0000","last_change":"2024-03-07T07:19:15.285743+0000","last_active":"2024-03-07T07:19:17.296626+0000","last_peered":"2024-03-07T07:19:17.296626+0000","last_clean":"2024-03-07T07:19:17.296626+0000","last_became_active":"2024-03-07T07:19:15.285484+0000","last_became_peered":"2024-03-07T07:19:15.285484+0000","last_unstale":"2024-03-07T07:19:17.296626+0000","last_undegraded":"2024-03-07T07:19:17.296626+0000","last_fullsized":"2024-03-07T07:19:17.296626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009544+0000","last_change":"2024-03-07T07:19:15.294301+0000","last_active":"2024-03-07T07:19:18.009544+0000","last_peered":"2024-03-07T07:19:18.009544+0000","last_clean":"2024-03-07T07:19:18.009544+0000","last_became_active":"2024-03-07T07:19:15.294023+0000","last_became_peered":"2024-03-07T07:19:15.294023+0000","last_unstale":"2024-03-07T07:19:18.009544+0000","last_undegraded":"2024-03-07T07:19:18.009544+0000","last_fullsized":"2024-03-07T07:19:18.009544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009524+0000","last_change":"2024-03-07T07:19:16.299031+0000","last_active":"2024-03-07T07:19:18.009524+0000","last_peered":"2024-03-07T07:19:18.009524+0000","last_clean":"2024-03-07T07:19:18.009524+0000","last_became_active":"2024-03-07T07:19:16.298760+0000","last_became_peered":"2024-03-07T07:19:16.298760+0000","last_unstale":"2024-03-07T07:19:18.009524+0000","last_undegraded":"2024-03-07T07:19:18.009524+0000","last_fullsized":"2024-03-07T07:19:18.009524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417339+0000","last_change":"2024-03-07T07:19:15.292620+0000","last_active":"2024-03-07T07:19:17.417339+0000","last_peered":"2024-03-07T07:19:17.417339+0000","last_clean":"2024-03-07T07:19:17.417339+0000","last_became_active":"2024-03-07T07:19:15.292337+0000","last_became_peered":"2024-03-07T07:19:15.292337+0000","last_unstale":"2024-03-07T07:19:17.417339+0000","last_undegraded":"2024-03-07T07:19:17.417339+0000","last_fullsized":"2024-03-07T07:19:17.417339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293989+0000","last_change":"2024-03-07T07:19:16.297970+0000","last_active":"2024-03-07T07:19:17.293989+0000","last_peered":"2024-03-07T07:19:17.293989+0000","last_clean":"2024-03-07T07:19:17.293989+0000","last_became_active":"2024-03-07T07:19:16.297661+0000","last_became_peered":"2024-03-07T07:19:16.297661+0000","last_unstale":"2024-03-07T07:19:17.293989+0000","last_undegraded":"2024-03-07T07:19:17.293989+0000","last_fullsized":"2024-03-07T07:19:17.293989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296514+0000","last_change":"2024-03-07T07:19:15.295817+0000","last_active":"2024-03-07T07:19:17.296514+0000","last_peered":"2024-03-07T07:19:17.296514+0000","last_clean":"2024-03-07T07:19:17.296514+0000","last_became_active":"2024-03-07T07:19:15.295538+0000","last_became_peered":"2024-03-07T07:19:15.295538+0000","last_unstale":"2024-03-07T07:19:17.296514+0000","last_undegraded":"2024-03-07T07:19:17.296514+0000","last_fullsized":"2024-03-07T07:19:17.296514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417710+0000","last_change":"2024-03-07T07:19:16.307152+0000","last_active":"2024-03-07T07:19:17.417710+0000","last_peered":"2024-03-07T07:19:17.417710+0000","last_clean":"2024-03-07T07:19:17.417710+0000","last_became_active":"2024-03-07T07:19:16.306890+0000","last_became_peered":"2024-03-07T07:19:16.306890+0000","last_unstale":"2024-03-07T07:19:17.417710+0000","last_undegraded":"2024-03-07T07:19:17.417710+0000","last_fullsized":"2024-03-07T07:19:17.417710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417582+0000","last_change":"2024-03-07T07:19:15.301303+0000","last_active":"2024-03-07T07:19:17.417582+0000","last_peered":"2024-03-07T07:19:17.417582+0000","last_clean":"2024-03-07T07:19:17.417582+0000","last_became_active":"2024-03-07T07:19:15.301031+0000","last_became_peered":"2024-03-07T07:19:15.301031+0000","last_unstale":"2024-03-07T07:19:17.417582+0000","last_undegraded":"2024-03-07T07:19:17.417582+0000","last_fullsized":"2024-03-07T07:19:17.417582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870348+0000","last_change":"2024-03-07T07:19:16.305499+0000","last_active":"2024-03-07T07:19:17.870348+0000","last_peered":"2024-03-07T07:19:17.870348+0000","last_clean":"2024-03-07T07:19:17.870348+0000","last_became_active":"2024-03-07T07:19:16.304971+0000","last_became_peered":"2024-03-07T07:19:16.304971+0000","last_unstale":"2024-03-07T07:19:17.870348+0000","last_undegraded":"2024-03-07T07:19:17.870348+0000","last_fullsized":"2024-03-07T07:19:17.870348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294183+0000","last_change":"2024-03-07T07:19:15.293364+0000","last_active":"2024-03-07T07:19:17.294183+0000","last_peered":"2024-03-07T07:19:17.294183+0000","last_clean":"2024-03-07T07:19:17.294183+0000","last_became_active":"2024-03-07T07:19:15.293069+0000","last_became_peered":"2024-03-07T07:19:15.293069+0000","last_unstale":"2024-03-07T07:19:17.294183+0000","last_undegraded":"2024-03-07T07:19:17.294183+0000","last_fullsized":"2024-03-07T07:19:17.294183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297511+0000","last_change":"2024-03-07T07:19:16.304722+0000","last_active":"2024-03-07T07:19:17.297511+0000","last_peered":"2024-03-07T07:19:17.297511+0000","last_clean":"2024-03-07T07:19:17.297511+0000","last_became_active":"2024-03-07T07:19:16.304438+0000","last_became_peered":"2024-03-07T07:19:16.304438+0000","last_unstale":"2024-03-07T07:19:17.297511+0000","last_undegraded":"2024-03-07T07:19:17.297511+0000","last_fullsized":"2024-03-07T07:19:17.297511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296596+0000","last_change":"2024-03-07T07:19:15.292852+0000","last_active":"2024-03-07T07:19:17.296596+0000","last_peered":"2024-03-07T07:19:17.296596+0000","last_clean":"2024-03-07T07:19:17.296596+0000","last_became_active":"2024-03-07T07:19:15.292584+0000","last_became_peered":"2024-03-07T07:19:15.292584+0000","last_unstale":"2024-03-07T07:19:17.296596+0000","last_undegraded":"2024-03-07T07:19:17.296596+0000","last_fullsized":"2024-03-07T07:19:17.296596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068903+0000","last_change":"2024-03-07T07:19:16.303347+0000","last_active":"2024-03-07T07:19:18.068903+0000","last_peered":"2024-03-07T07:19:18.068903+0000","last_clean":"2024-03-07T07:19:18.068903+0000","last_became_active":"2024-03-07T07:19:16.303083+0000","last_became_peered":"2024-03-07T07:19:16.303083+0000","last_unstale":"2024-03-07T07:19:18.068903+0000","last_undegraded":"2024-03-07T07:19:18.068903+0000","last_fullsized":"2024-03-07T07:19:18.068903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870639+0000","last_change":"2024-03-07T07:19:15.296780+0000","last_active":"2024-03-07T07:19:17.870639+0000","last_peered":"2024-03-07T07:19:17.870639+0000","last_clean":"2024-03-07T07:19:17.870639+0000","last_became_active":"2024-03-07T07:19:15.296300+0000","last_became_peered":"2024-03-07T07:19:15.296300+0000","last_unstale":"2024-03-07T07:19:17.870639+0000","last_undegraded":"2024-03-07T07:19:17.870639+0000","last_fullsized":"2024-03-07T07:19:17.870639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870345+0000","last_change":"2024-03-07T07:19:16.297140+0000","last_active":"2024-03-07T07:19:17.870345+0000","last_peered":"2024-03-07T07:19:17.870345+0000","last_clean":"2024-03-07T07:19:17.870345+0000","last_became_active":"2024-03-07T07:19:16.296854+0000","last_became_peered":"2024-03-07T07:19:16.296854+0000","last_unstale":"2024-03-07T07:19:17.870345+0000","last_undegraded":"2024-03-07T07:19:17.870345+0000","last_fullsized":"2024-03-07T07:19:17.870345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417473+0000","last_change":"2024-03-07T07:19:15.299975+0000","last_active":"2024-03-07T07:19:17.417473+0000","last_peered":"2024-03-07T07:19:17.417473+0000","last_clean":"2024-03-07T07:19:17.417473+0000","last_became_active":"2024-03-07T07:19:15.299753+0000","last_became_peered":"2024-03-07T07:19:15.299753+0000","last_unstale":"2024-03-07T07:19:17.417473+0000","last_undegraded":"2024-03-07T07:19:17.417473+0000","last_fullsized":"2024-03-07T07:19:17.417473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068921+0000","last_change":"2024-03-07T07:19:16.307214+0000","last_active":"2024-03-07T07:19:18.068921+0000","last_peered":"2024-03-07T07:19:18.068921+0000","last_clean":"2024-03-07T07:19:18.068921+0000","last_became_active":"2024-03-07T07:19:16.306690+0000","last_became_peered":"2024-03-07T07:19:16.306690+0000","last_unstale":"2024-03-07T07:19:18.068921+0000","last_undegraded":"2024-03-07T07:19:18.068921+0000","last_fullsized":"2024-03-07T07:19:18.068921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296268+0000","last_change":"2024-03-07T07:19:15.286950+0000","last_active":"2024-03-07T07:19:17.296268+0000","last_peered":"2024-03-07T07:19:17.296268+0000","last_clean":"2024-03-07T07:19:17.296268+0000","last_became_active":"2024-03-07T07:19:15.286655+0000","last_became_peered":"2024-03-07T07:19:15.286655+0000","last_unstale":"2024-03-07T07:19:17.296268+0000","last_undegraded":"2024-03-07T07:19:17.296268+0000","last_fullsized":"2024-03-07T07:19:17.296268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296267+0000","last_change":"2024-03-07T07:19:16.299235+0000","last_active":"2024-03-07T07:19:17.296267+0000","last_peered":"2024-03-07T07:19:17.296267+0000","last_clean":"2024-03-07T07:19:17.296267+0000","last_became_active":"2024-03-07T07:19:16.298945+0000","last_became_peered":"2024-03-07T07:19:16.298945+0000","last_unstale":"2024-03-07T07:19:17.296267+0000","last_undegraded":"2024-03-07T07:19:17.296267+0000","last_fullsized":"2024-03-07T07:19:17.296267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068707+0000","last_change":"2024-03-07T07:19:15.289721+0000","last_active":"2024-03-07T07:19:18.068707+0000","last_peered":"2024-03-07T07:19:18.068707+0000","last_clean":"2024-03-07T07:19:18.068707+0000","last_became_active":"2024-03-07T07:19:15.289410+0000","last_became_peered":"2024-03-07T07:19:15.289410+0000","last_unstale":"2024-03-07T07:19:18.068707+0000","last_undegraded":"2024-03-07T07:19:18.068707+0000","last_fullsized":"2024-03-07T07:19:18.068707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068748+0000","last_change":"2024-03-07T07:19:16.306954+0000","last_active":"2024-03-07T07:19:18.068748+0000","last_peered":"2024-03-07T07:19:18.068748+0000","last_clean":"2024-03-07T07:19:18.068748+0000","last_became_active":"2024-03-07T07:19:16.306639+0000","last_became_peered":"2024-03-07T07:19:16.306639+0000","last_unstale":"2024-03-07T07:19:18.068748+0000","last_undegraded":"2024-03-07T07:19:18.068748+0000","last_fullsized":"2024-03-07T07:19:18.068748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297392+0000","last_change":"2024-03-07T07:19:15.297356+0000","last_active":"2024-03-07T07:19:17.297392+0000","last_peered":"2024-03-07T07:19:17.297392+0000","last_clean":"2024-03-07T07:19:17.297392+0000","last_became_active":"2024-03-07T07:19:15.296989+0000","last_became_peered":"2024-03-07T07:19:15.296989+0000","last_unstale":"2024-03-07T07:19:17.297392+0000","last_undegraded":"2024-03-07T07:19:17.297392+0000","last_fullsized":"2024-03-07T07:19:17.297392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:17:57.790720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069126+0000","last_change":"2024-03-07T07:19:16.306845+0000","last_active":"2024-03-07T07:19:18.069126+0000","last_peered":"2024-03-07T07:19:18.069126+0000","last_clean":"2024-03-07T07:19:18.069126+0000","last_became_active":"2024-03-07T07:19:16.306406+0000","last_became_peered":"2024-03-07T07:19:16.306406+0000","last_unstale":"2024-03-07T07:19:18.069126+0000","last_undegraded":"2024-03-07T07:19:18.069126+0000","last_fullsized":"2024-03-07T07:19:18.069126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068846+0000","last_change":"2024-03-07T07:19:15.293365+0000","last_active":"2024-03-07T07:19:18.068846+0000","last_peered":"2024-03-07T07:19:18.068846+0000","last_clean":"2024-03-07T07:19:18.068846+0000","last_became_active":"2024-03-07T07:19:15.293062+0000","last_became_peered":"2024-03-07T07:19:15.293062+0000","last_unstale":"2024-03-07T07:19:18.068846+0000","last_undegraded":"2024-03-07T07:19:18.068846+0000","last_fullsized":"2024-03-07T07:19:18.068846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069160+0000","last_change":"2024-03-07T07:19:16.307514+0000","last_active":"2024-03-07T07:19:18.069160+0000","last_peered":"2024-03-07T07:19:18.069160+0000","last_clean":"2024-03-07T07:19:18.069160+0000","last_became_active":"2024-03-07T07:19:16.307226+0000","last_became_peered":"2024-03-07T07:19:16.307226+0000","last_unstale":"2024-03-07T07:19:18.069160+0000","last_undegraded":"2024-03-07T07:19:18.069160+0000","last_fullsized":"2024-03-07T07:19:18.069160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:25:12.269915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294284+0000","last_change":"2024-03-07T07:19:15.290118+0000","last_active":"2024-03-07T07:19:17.294284+0000","last_peered":"2024-03-07T07:19:17.294284+0000","last_clean":"2024-03-07T07:19:17.294284+0000","last_became_active":"2024-03-07T07:19:15.289812+0000","last_became_peered":"2024-03-07T07:19:15.289812+0000","last_unstale":"2024-03-07T07:19:17.294284+0000","last_undegraded":"2024-03-07T07:19:17.294284+0000","last_fullsized":"2024-03-07T07:19:17.294284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417670+0000","last_change":"2024-03-07T07:19:16.303739+0000","last_active":"2024-03-07T07:19:17.417670+0000","last_peered":"2024-03-07T07:19:17.417670+0000","last_clean":"2024-03-07T07:19:17.417670+0000","last_became_active":"2024-03-07T07:19:16.303279+0000","last_became_peered":"2024-03-07T07:19:16.303279+0000","last_unstale":"2024-03-07T07:19:17.417670+0000","last_undegraded":"2024-03-07T07:19:17.417670+0000","last_fullsized":"2024-03-07T07:19:17.417670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068866+0000","last_change":"2024-03-07T07:19:15.296198+0000","last_active":"2024-03-07T07:19:18.068866+0000","last_peered":"2024-03-07T07:19:18.068866+0000","last_clean":"2024-03-07T07:19:18.068866+0000","last_became_active":"2024-03-07T07:19:15.295886+0000","last_became_peered":"2024-03-07T07:19:15.295886+0000","last_unstale":"2024-03-07T07:19:18.068866+0000","last_undegraded":"2024-03-07T07:19:18.068866+0000","last_fullsized":"2024-03-07T07:19:18.068866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069075+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:18.069075+0000","last_peered":"2024-03-07T07:19:18.069075+0000","last_clean":"2024-03-07T07:19:18.069075+0000","last_became_active":"2024-03-07T07:19:16.307119+0000","last_became_peered":"2024-03-07T07:19:16.307119+0000","last_unstale":"2024-03-07T07:19:18.069075+0000","last_undegraded":"2024-03-07T07:19:18.069075+0000","last_fullsized":"2024-03-07T07:19:18.069075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068695+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:19:18.068695+0000","last_peered":"2024-03-07T07:19:18.068695+0000","last_clean":"2024-03-07T07:19:18.068695+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:19:18.068695+0000","last_undegraded":"2024-03-07T07:19:18.068695+0000","last_fullsized":"2024-03-07T07:19:18.068695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417348+0000","last_change":"2024-03-07T07:19:15.295100+0000","last_active":"2024-03-07T07:19:17.417348+0000","last_peered":"2024-03-07T07:19:17.417348+0000","last_clean":"2024-03-07T07:19:17.417348+0000","last_became_active":"2024-03-07T07:19:15.294643+0000","last_became_peered":"2024-03-07T07:19:15.294643+0000","last_unstale":"2024-03-07T07:19:17.417348+0000","last_undegraded":"2024-03-07T07:19:17.417348+0000","last_fullsized":"2024-03-07T07:19:17.417348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.306837+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.305695+0000","last_became_peered":"2024-03-07T07:19:16.305695+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068642+0000","last_change":"2024-03-07T07:19:15.301442+0000","last_active":"2024-03-07T07:19:18.068642+0000","last_peered":"2024-03-07T07:19:18.068642+0000","last_clean":"2024-03-07T07:19:18.068642+0000","last_became_active":"2024-03-07T07:19:15.300526+0000","last_became_peered":"2024-03-07T07:19:15.300526+0000","last_unstale":"2024-03-07T07:19:18.068642+0000","last_undegraded":"2024-03-07T07:19:18.068642+0000","last_fullsized":"2024-03-07T07:19:18.068642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870083+0000","last_change":"2024-03-07T07:19:16.297399+0000","last_active":"2024-03-07T07:19:17.870083+0000","last_peered":"2024-03-07T07:19:17.870083+0000","last_clean":"2024-03-07T07:19:17.870083+0000","last_became_active":"2024-03-07T07:19:16.296941+0000","last_became_peered":"2024-03-07T07:19:16.296941+0000","last_unstale":"2024-03-07T07:19:17.870083+0000","last_undegraded":"2024-03-07T07:19:17.870083+0000","last_fullsized":"2024-03-07T07:19:17.870083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:10.125663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068824+0000","last_change":"2024-03-07T07:19:15.291261+0000","last_active":"2024-03-07T07:19:18.068824+0000","last_peered":"2024-03-07T07:19:18.068824+0000","last_clean":"2024-03-07T07:19:18.068824+0000","last_became_active":"2024-03-07T07:19:15.290977+0000","last_became_peered":"2024-03-07T07:19:15.290977+0000","last_unstale":"2024-03-07T07:19:18.068824+0000","last_undegraded":"2024-03-07T07:19:18.068824+0000","last_fullsized":"2024-03-07T07:19:18.068824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294009+0000","last_change":"2024-03-07T07:19:16.298068+0000","last_active":"2024-03-07T07:19:17.294009+0000","last_peered":"2024-03-07T07:19:17.294009+0000","last_clean":"2024-03-07T07:19:17.294009+0000","last_became_active":"2024-03-07T07:19:16.297795+0000","last_became_peered":"2024-03-07T07:19:16.297795+0000","last_unstale":"2024-03-07T07:19:17.294009+0000","last_undegraded":"2024-03-07T07:19:17.294009+0000","last_fullsized":"2024-03-07T07:19:17.294009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417588+0000","last_change":"2024-03-07T07:19:15.296795+0000","last_active":"2024-03-07T07:19:17.417588+0000","last_peered":"2024-03-07T07:19:17.417588+0000","last_clean":"2024-03-07T07:19:17.417588+0000","last_became_active":"2024-03-07T07:19:15.296534+0000","last_became_peered":"2024-03-07T07:19:15.296534+0000","last_unstale":"2024-03-07T07:19:17.417588+0000","last_undegraded":"2024-03-07T07:19:17.417588+0000","last_fullsized":"2024-03-07T07:19:17.417588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297568+0000","last_change":"2024-03-07T07:19:16.295673+0000","last_active":"2024-03-07T07:19:17.297568+0000","last_peered":"2024-03-07T07:19:17.297568+0000","last_clean":"2024-03-07T07:19:17.297568+0000","last_became_active":"2024-03-07T07:19:16.295139+0000","last_became_peered":"2024-03-07T07:19:16.295139+0000","last_unstale":"2024-03-07T07:19:17.297568+0000","last_undegraded":"2024-03-07T07:19:17.297568+0000","last_fullsized":"2024-03-07T07:19:17.297568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294148+0000","last_change":"2024-03-07T07:19:15.297274+0000","last_active":"2024-03-07T07:19:17.294148+0000","last_peered":"2024-03-07T07:19:17.294148+0000","last_clean":"2024-03-07T07:19:17.294148+0000","last_became_active":"2024-03-07T07:19:15.297000+0000","last_became_peered":"2024-03-07T07:19:15.297000+0000","last_unstale":"2024-03-07T07:19:17.294148+0000","last_undegraded":"2024-03-07T07:19:17.294148+0000","last_fullsized":"2024-03-07T07:19:17.294148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417617+0000","last_change":"2024-03-07T07:19:16.305048+0000","last_active":"2024-03-07T07:19:17.417617+0000","last_peered":"2024-03-07T07:19:17.417617+0000","last_clean":"2024-03-07T07:19:17.417617+0000","last_became_active":"2024-03-07T07:19:16.303993+0000","last_became_peered":"2024-03-07T07:19:16.303993+0000","last_unstale":"2024-03-07T07:19:17.417617+0000","last_undegraded":"2024-03-07T07:19:17.417617+0000","last_fullsized":"2024-03-07T07:19:17.417617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069397+0000","last_change":"2024-03-07T07:19:15.300909+0000","last_active":"2024-03-07T07:19:18.069397+0000","last_peered":"2024-03-07T07:19:18.069397+0000","last_clean":"2024-03-07T07:19:18.069397+0000","last_became_active":"2024-03-07T07:19:15.300573+0000","last_became_peered":"2024-03-07T07:19:15.300573+0000","last_unstale":"2024-03-07T07:19:18.069397+0000","last_undegraded":"2024-03-07T07:19:18.069397+0000","last_fullsized":"2024-03-07T07:19:18.069397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294190+0000","last_change":"2024-03-07T07:19:16.298525+0000","last_active":"2024-03-07T07:19:17.294190+0000","last_peered":"2024-03-07T07:19:17.294190+0000","last_clean":"2024-03-07T07:19:17.294190+0000","last_became_active":"2024-03-07T07:19:16.298264+0000","last_became_peered":"2024-03-07T07:19:16.298264+0000","last_unstale":"2024-03-07T07:19:17.294190+0000","last_undegraded":"2024-03-07T07:19:17.294190+0000","last_fullsized":"2024-03-07T07:19:17.294190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.295564+0000","last_change":"2024-03-07T07:19:15.291265+0000","last_active":"2024-03-07T07:19:17.295564+0000","last_peered":"2024-03-07T07:19:17.295564+0000","last_clean":"2024-03-07T07:19:17.295564+0000","last_became_active":"2024-03-07T07:19:15.290889+0000","last_became_peered":"2024-03-07T07:19:15.290889+0000","last_unstale":"2024-03-07T07:19:17.295564+0000","last_undegraded":"2024-03-07T07:19:17.295564+0000","last_fullsized":"2024-03-07T07:19:17.295564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009905+0000","last_change":"2024-03-07T07:19:16.303881+0000","last_active":"2024-03-07T07:19:18.009905+0000","last_peered":"2024-03-07T07:19:18.009905+0000","last_clean":"2024-03-07T07:19:18.009905+0000","last_became_active":"2024-03-07T07:19:16.303597+0000","last_became_peered":"2024-03-07T07:19:16.303597+0000","last_unstale":"2024-03-07T07:19:18.009905+0000","last_undegraded":"2024-03-07T07:19:18.009905+0000","last_fullsized":"2024-03-07T07:19:18.009905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296735+0000","last_change":"2024-03-07T07:19:15.296436+0000","last_active":"2024-03-07T07:19:17.296735+0000","last_peered":"2024-03-07T07:19:17.296735+0000","last_clean":"2024-03-07T07:19:17.296735+0000","last_became_active":"2024-03-07T07:19:15.296152+0000","last_became_peered":"2024-03-07T07:19:15.296152+0000","last_unstale":"2024-03-07T07:19:17.296735+0000","last_undegraded":"2024-03-07T07:19:17.296735+0000","last_fullsized":"2024-03-07T07:19:17.296735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870218+0000","last_change":"2024-03-07T07:19:16.300794+0000","last_active":"2024-03-07T07:19:17.870218+0000","last_peered":"2024-03-07T07:19:17.870218+0000","last_clean":"2024-03-07T07:19:17.870218+0000","last_became_active":"2024-03-07T07:19:16.300531+0000","last_became_peered":"2024-03-07T07:19:16.300531+0000","last_unstale":"2024-03-07T07:19:17.870218+0000","last_undegraded":"2024-03-07T07:19:17.870218+0000","last_fullsized":"2024-03-07T07:19:17.870218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068653+0000","last_change":"2024-03-07T07:19:15.301871+0000","last_active":"2024-03-07T07:19:18.068653+0000","last_peered":"2024-03-07T07:19:18.068653+0000","last_clean":"2024-03-07T07:19:18.068653+0000","last_became_active":"2024-03-07T07:19:15.301269+0000","last_became_peered":"2024-03-07T07:19:15.301269+0000","last_unstale":"2024-03-07T07:19:18.068653+0000","last_undegraded":"2024-03-07T07:19:18.068653+0000","last_fullsized":"2024-03-07T07:19:18.068653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297261+0000","last_change":"2024-03-07T07:19:16.293571+0000","last_active":"2024-03-07T07:19:17.297261+0000","last_peered":"2024-03-07T07:19:17.297261+0000","last_clean":"2024-03-07T07:19:17.297261+0000","last_became_active":"2024-03-07T07:19:16.293217+0000","last_became_peered":"2024-03-07T07:19:16.293217+0000","last_unstale":"2024-03-07T07:19:17.297261+0000","last_undegraded":"2024-03-07T07:19:17.297261+0000","last_fullsized":"2024-03-07T07:19:17.297261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:11.899192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294039+0000","last_change":"2024-03-07T07:19:15.291033+0000","last_active":"2024-03-07T07:19:17.294039+0000","last_peered":"2024-03-07T07:19:17.294039+0000","last_clean":"2024-03-07T07:19:17.294039+0000","last_became_active":"2024-03-07T07:19:15.290699+0000","last_became_peered":"2024-03-07T07:19:15.290699+0000","last_unstale":"2024-03-07T07:19:17.294039+0000","last_undegraded":"2024-03-07T07:19:17.294039+0000","last_fullsized":"2024-03-07T07:19:17.294039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009889+0000","last_change":"2024-03-07T07:19:16.300154+0000","last_active":"2024-03-07T07:19:18.009889+0000","last_peered":"2024-03-07T07:19:18.009889+0000","last_clean":"2024-03-07T07:19:18.009889+0000","last_became_active":"2024-03-07T07:19:16.299831+0000","last_became_peered":"2024-03-07T07:19:16.299831+0000","last_unstale":"2024-03-07T07:19:18.009889+0000","last_undegraded":"2024-03-07T07:19:18.009889+0000","last_fullsized":"2024-03-07T07:19:18.009889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294117+0000","last_change":"2024-03-07T07:19:15.292788+0000","last_active":"2024-03-07T07:19:17.294117+0000","last_peered":"2024-03-07T07:19:17.294117+0000","last_clean":"2024-03-07T07:19:17.294117+0000","last_became_active":"2024-03-07T07:19:15.292478+0000","last_became_peered":"2024-03-07T07:19:15.292478+0000","last_unstale":"2024-03-07T07:19:17.294117+0000","last_undegraded":"2024-03-07T07:19:17.294117+0000","last_fullsized":"2024-03-07T07:19:17.294117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.302180+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069079+0000","last_change":"2024-03-07T07:19:15.300817+0000","last_active":"2024-03-07T07:19:18.069079+0000","last_peered":"2024-03-07T07:19:18.069079+0000","last_clean":"2024-03-07T07:19:18.069079+0000","last_became_active":"2024-03-07T07:19:15.300502+0000","last_became_peered":"2024-03-07T07:19:15.300502+0000","last_unstale":"2024-03-07T07:19:18.069079+0000","last_undegraded":"2024-03-07T07:19:18.069079+0000","last_fullsized":"2024-03-07T07:19:18.069079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296526+0000","last_change":"2024-03-07T07:19:16.300613+0000","last_active":"2024-03-07T07:19:17.296526+0000","last_peered":"2024-03-07T07:19:17.296526+0000","last_clean":"2024-03-07T07:19:17.296526+0000","last_became_active":"2024-03-07T07:19:16.299969+0000","last_became_peered":"2024-03-07T07:19:16.299969+0000","last_unstale":"2024-03-07T07:19:17.296526+0000","last_undegraded":"2024-03-07T07:19:17.296526+0000","last_fullsized":"2024-03-07T07:19:17.296526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417649+0000","last_change":"2024-03-07T07:19:16.316583+0000","last_active":"2024-03-07T07:19:17.417649+0000","last_peered":"2024-03-07T07:19:17.417649+0000","last_clean":"2024-03-07T07:19:17.417649+0000","last_became_active":"2024-03-07T07:19:16.316351+0000","last_became_peered":"2024-03-07T07:19:16.316351+0000","last_unstale":"2024-03-07T07:19:17.417649+0000","last_undegraded":"2024-03-07T07:19:17.417649+0000","last_fullsized":"2024-03-07T07:19:17.417649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069099+0000","last_change":"2024-03-07T07:19:15.300879+0000","last_active":"2024-03-07T07:19:18.069099+0000","last_peered":"2024-03-07T07:19:18.069099+0000","last_clean":"2024-03-07T07:19:18.069099+0000","last_became_active":"2024-03-07T07:19:15.300594+0000","last_became_peered":"2024-03-07T07:19:15.300594+0000","last_unstale":"2024-03-07T07:19:18.069099+0000","last_undegraded":"2024-03-07T07:19:18.069099+0000","last_fullsized":"2024-03-07T07:19:18.069099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417463+0000","last_change":"2024-03-07T07:19:16.313560+0000","last_active":"2024-03-07T07:19:17.417463+0000","last_peered":"2024-03-07T07:19:17.417463+0000","last_clean":"2024-03-07T07:19:17.417463+0000","last_became_active":"2024-03-07T07:19:16.313130+0000","last_became_peered":"2024-03-07T07:19:16.313130+0000","last_unstale":"2024-03-07T07:19:17.417463+0000","last_undegraded":"2024-03-07T07:19:17.417463+0000","last_fullsized":"2024-03-07T07:19:17.417463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069186+0000","last_change":"2024-03-07T07:19:15.301563+0000","last_active":"2024-03-07T07:19:18.069186+0000","last_peered":"2024-03-07T07:19:18.069186+0000","last_clean":"2024-03-07T07:19:18.069186+0000","last_became_active":"2024-03-07T07:19:15.301282+0000","last_became_peered":"2024-03-07T07:19:15.301282+0000","last_unstale":"2024-03-07T07:19:18.069186+0000","last_undegraded":"2024-03-07T07:19:18.069186+0000","last_fullsized":"2024-03-07T07:19:18.069186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870211+0000","last_change":"2024-03-07T07:19:16.306438+0000","last_active":"2024-03-07T07:19:17.870211+0000","last_peered":"2024-03-07T07:19:17.870211+0000","last_clean":"2024-03-07T07:19:17.870211+0000","last_became_active":"2024-03-07T07:19:16.305209+0000","last_became_peered":"2024-03-07T07:19:16.305209+0000","last_unstale":"2024-03-07T07:19:17.870211+0000","last_undegraded":"2024-03-07T07:19:17.870211+0000","last_fullsized":"2024-03-07T07:19:17.870211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009925+0000","last_change":"2024-03-07T07:19:15.299866+0000","last_active":"2024-03-07T07:19:18.009925+0000","last_peered":"2024-03-07T07:19:18.009925+0000","last_clean":"2024-03-07T07:19:18.009925+0000","last_became_active":"2024-03-07T07:19:15.299621+0000","last_became_peered":"2024-03-07T07:19:15.299621+0000","last_unstale":"2024-03-07T07:19:18.009925+0000","last_undegraded":"2024-03-07T07:19:18.009925+0000","last_fullsized":"2024-03-07T07:19:18.009925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009615+0000","last_change":"2024-03-07T07:19:16.296975+0000","last_active":"2024-03-07T07:19:18.009615+0000","last_peered":"2024-03-07T07:19:18.009615+0000","last_clean":"2024-03-07T07:19:18.009615+0000","last_became_active":"2024-03-07T07:19:16.296702+0000","last_became_peered":"2024-03-07T07:19:16.296702+0000","last_unstale":"2024-03-07T07:19:18.009615+0000","last_undegraded":"2024-03-07T07:19:18.009615+0000","last_fullsized":"2024-03-07T07:19:18.009615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870446+0000","last_change":"2024-03-07T07:19:15.303925+0000","last_active":"2024-03-07T07:19:17.870446+0000","last_peered":"2024-03-07T07:19:17.870446+0000","last_clean":"2024-03-07T07:19:17.870446+0000","last_became_active":"2024-03-07T07:19:15.303646+0000","last_became_peered":"2024-03-07T07:19:15.303646+0000","last_unstale":"2024-03-07T07:19:17.870446+0000","last_undegraded":"2024-03-07T07:19:17.870446+0000","last_fullsized":"2024-03-07T07:19:17.870446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:19:24.430 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-07T07:19:24.430 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-07T07:19:24.430 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-07T07:19:24.430 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-07T07:19:24.431 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-07T07:19:24.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.568+0000 7f14b2087640 1 Processor -- start 2024-03-07T07:19:24.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.568+0000 7f14b2087640 1 -- start start 2024-03-07T07:19:24.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14b2087640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14ac0b9350 0x7f14ac15e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14b2087640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f14ac15ecf0 0x7f14ac142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14b2087640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac0b6440 con 0x7f14ac141370 2024-03-07T07:19:24.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14b2087640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac15f150 con 0x7f14ac0b9350 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14b2087640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac141980 con 0x7f14ac15ecf0 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14aaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14ac0b9350 0x7f14ac15e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14aaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f14ac0b9350 0x7f14ac15e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34230/0 (socket says 172.21.15.64:34230) 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14ab7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f14ac141370 0x7f14ac0b8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48650/0 (socket says 172.21.15.64:48650) 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14aaffd640 1 -- 172.21.15.64:0/2658344587 learned_addr learned my addr 172.21.15.64:0/2658344587 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14abfff640 1 --2- 172.21.15.64:0/2658344587 >> v2:172.21.15.120:3300/0 conn(0x7f14ac15ecf0 0x7f14ac142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14aaffd640 1 -- 172.21.15.64:0/2658344587 >> v2:172.21.15.120:3300/0 conn(0x7f14ac15ecf0 msgr2=0x7f14ac142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14aaffd640 1 --2- 172.21.15.64:0/2658344587 >> v2:172.21.15.120:3300/0 conn(0x7f14ac15ecf0 0x7f14ac142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14aaffd640 1 -- 172.21.15.64:0/2658344587 >> v1:172.21.15.64:6789/0 conn(0x7f14ac141370 legacy=0x7f14ac0b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.569+0000 7f14aaffd640 1 -- 172.21.15.64:0/2658344587 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac15f3f0 con 0x7f14ac0b9350 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.570+0000 7f14aaffd640 1 --2- 172.21.15.64:0/2658344587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac0b9350 0x7f14ac15e7b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14ac15c3f0 tx=0x7f14a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ad1ab43ab5d38d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.570+0000 7f14aa7fc640 1 -- 172.21.15.64:0/2658344587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0035020 con 0x7f14ac0b9350 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.570+0000 7f14aa7fc640 1 -- 172.21.15.64:0/2658344587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a0031870 con 0x7f14ac0b9350 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.570+0000 7f14aa7fc640 1 -- 172.21.15.64:0/2658344587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0031c40 con 0x7f14ac0b9350 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 -- 172.21.15.64:0/2658344587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac0b9350 msgr2=0x7f14ac15e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 --2- 172.21.15.64:0/2658344587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac0b9350 0x7f14ac15e7b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14ac15c3f0 tx=0x7f14a002b910 comp rx=0 tx=0).stop 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 -- 172.21.15.64:0/2658344587 shutdown_connections 2024-03-07T07:19:24.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 --2- 172.21.15.64:0/2658344587 >> v2:172.21.15.120:3300/0 conn(0x7f14ac15ecf0 0x7f14ac142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 --2- 172.21.15.64:0/2658344587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac0b9350 0x7f14ac15e7b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 -- 172.21.15.64:0/2658344587 >> 172.21.15.64:0/2658344587 conn(0x7f14ac072fc0 msgr2=0x7f14ac0b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:24.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 -- 172.21.15.64:0/2658344587 shutdown_connections 2024-03-07T07:19:24.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.571+0000 7f14b2087640 1 -- 172.21.15.64:0/2658344587 wait complete. 2024-03-07T07:19:24.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.572+0000 7f14b2087640 1 Processor -- start 2024-03-07T07:19:24.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.572+0000 7f14b2087640 1 -- start start 2024-03-07T07:19:24.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14b2087640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14ac0b9350 0x7f14ac15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14b2087640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14ac141370 0x7f14ac13c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14b2087640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac15ecf0 0x7f14ac13f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14b2087640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac142980 con 0x7f14ac15ecf0 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14b2087640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac141c40 con 0x7f14ac0b9350 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14b2087640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac142b00 con 0x7f14ac141370 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac15ecf0 0x7f14ac13f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac15ecf0 0x7f14ac13f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34244/0 (socket says 172.21.15.64:34244) 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 -- 172.21.15.64:0/1142373922 learned_addr learned my addr 172.21.15.64:0/1142373922 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14ab7fe640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14ac0b9350 0x7f14ac15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14aaffd640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14ac141370 0x7f14ac13c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 -- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14ac141370 msgr2=0x7f14ac13c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14ac141370 0x7f14ac13c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 -- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14ac0b9350 msgr2=0x7f14ac15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14ac0b9350 0x7f14ac15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 -- 172.21.15.64:0/1142373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac158ef0 con 0x7f14ac15ecf0 2024-03-07T07:19:24.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14ab7fe640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14ac0b9350 0x7f14ac15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f14abfff640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac15ecf0 0x7f14ac13f180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009d30 tx=0x7f14a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.573+0000 7f1493fff640 1 -- 172.21.15.64:0/1142373922 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4018020 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.574+0000 7f1493fff640 1 -- 172.21.15.64:0/1142373922 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4014480 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.574+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac1591d0 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.574+0000 7f1493fff640 1 -- 172.21.15.64:0/1142373922 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4014870 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.574+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac13f6c0 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.574+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac062c90 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.575+0000 7f1493fff640 1 -- 172.21.15.64:0/1142373922 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f14a4005d10 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.575+0000 7f1493fff640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f149806b4d0 0x7f149806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.575+0000 7f1493fff640 1 -- 172.21.15.64:0/1142373922 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14a408ab30 con 0x7f14ac15ecf0 2024-03-07T07:19:24.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.576+0000 7f14ab7fe640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f149806b4d0 0x7f149806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.576+0000 7f14ab7fe640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f149806b4d0 0x7f149806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f149c007fd0 tx=0x7f149c007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.580+0000 7f1493fff640 1 -- 172.21.15.64:0/1142373922 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14a4022070 con 0x7f14ac15ecf0 2024-03-07T07:19:24.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.806+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f14ac0b7200 con 0x7f14ac15ecf0 2024-03-07T07:19:24.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.809+0000 7f1493fff640 1 -- 172.21.15.64:0/1142373922 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171787 (secure 0 0 0) 0x7f14a4010040 con 0x7f14ac15ecf0 2024-03-07T07:19:24.811 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:24.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f149806b4d0 msgr2=0x7f149806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f149806b4d0 0x7f149806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f149c007fd0 tx=0x7f149c007e80 comp rx=0 tx=0).stop 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac15ecf0 msgr2=0x7f14ac13f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac15ecf0 0x7f14ac13f180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009d30 tx=0x7f14a400c490 comp rx=0 tx=0).stop 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 shutdown_connections 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f149806b4d0 0x7f149806d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14ac15ecf0 0x7f14ac13f180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14ac141370 0x7f14ac13c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 --2- 172.21.15.64:0/1142373922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14ac0b9350 0x7f14ac15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 >> 172.21.15.64:0/1142373922 conn(0x7f14ac072fc0 msgr2=0x7f14ac144220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 shutdown_connections 2024-03-07T07:19:24.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.812+0000 7f14b2087640 1 -- 172.21.15.64:0/1142373922 wait complete. 2024-03-07T07:19:24.822 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":182409167},{"type":"v1","addr":"172.21.15.120:6833","nonce":182409167}]},"active_addr":"172.21.15.120:6833/182409167","active_change":"2024-03-07T07:18:56.167510+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2257167035}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":1215788341}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":2653562423}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.120:0","nonce":3707604984}]}]} 2024-03-07T07:19:24.825 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-07T07:19:24.825 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-07T07:19:24.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.927+0000 7fa8cdfa7640 1 Processor -- start 2024-03-07T07:19:24.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.927+0000 7fa8cdfa7640 1 -- start start 2024-03-07T07:19:24.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8cdfa7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa8c81412e0 0x7fa8c80b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8cdfa7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c8070400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8cdfa7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa8c8070940 0x7fa8c815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8cdfa7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c8071c90 con 0x7fa8c81412e0 2024-03-07T07:19:24.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8cdfa7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c80b3120 con 0x7fa8c8070940 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8cdfa7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8070dd0 con 0x7fa8c80b2ca0 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c77fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa8c81412e0 0x7fa8c80b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c7fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa8c8070940 0x7fa8c815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c6ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c8070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c77fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa8c81412e0 0x7fa8c80b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34246/0 (socket says 172.21.15.64:34246) 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c77fe640 1 -- 172.21.15.64:0/1409670156 learned_addr learned my addr 172.21.15.64:0/1409670156 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/1409670156 >> v2:172.21.15.120:3300/0 conn(0x7fa8c8070940 0x7fa8c815ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34068/0 (socket says 172.21.15.64:34068) 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c7fff640 1 -- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 msgr2=0x7fa8c8070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c8070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c7fff640 1 -- 172.21.15.64:0/1409670156 >> v2:172.21.15.64:3300/0 conn(0x7fa8c81412e0 msgr2=0x7fa8c80b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/1409670156 >> v2:172.21.15.64:3300/0 conn(0x7fa8c81412e0 0x7fa8c80b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c7fff640 1 -- 172.21.15.64:0/1409670156 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8071080 con 0x7fa8c8070940 2024-03-07T07:19:24.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.928+0000 7fa8c6ffd640 1 --2- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c8070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:24.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.929+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 0x7fa8c815ea40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8c806eeb0 tx=0x7fa8c002bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=677ac3efbbc1e41e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.929+0000 7fa8c67fc640 1 -- 172.21.15.64:0/1409670156 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0039070 con 0x7fa8c8070940 2024-03-07T07:19:24.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.929+0000 7fa8c67fc640 1 -- 172.21.15.64:0/1409670156 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c002cab0 con 0x7fa8c8070940 2024-03-07T07:19:24.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.929+0000 7fa8c67fc640 1 -- 172.21.15.64:0/1409670156 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0034440 con 0x7fa8c8070940 2024-03-07T07:19:24.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 msgr2=0x7fa8c815ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 0x7fa8c815ea40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8c806eeb0 tx=0x7fa8c002bd00 comp rx=0 tx=0).stop 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/1409670156 shutdown_connections 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 0x7fa8c815ea40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/1409670156 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c8070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/1409670156 >> v2:172.21.15.64:3300/0 conn(0x7fa8c81412e0 0x7fa8c80b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/1409670156 >> 172.21.15.64:0/1409670156 conn(0x7fa8c8072490 msgr2=0x7fa8c8072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/1409670156 shutdown_connections 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.930+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/1409670156 wait complete. 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.931+0000 7fa8cdfa7640 1 Processor -- start 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.931+0000 7fa8cdfa7640 1 -- start start 2024-03-07T07:19:24.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8cdfa7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 0x7fa8c8142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8cdfa7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8cdfa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8c81412e0 0x7fa8c81441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8cdfa7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8071920 con 0x7fa8c81412e0 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8cdfa7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8071640 con 0x7fa8c8070940 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8cdfa7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c815ef80 con 0x7fa8c80b2ca0 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8c81412e0 0x7fa8c81441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8c81412e0 0x7fa8c81441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34252/0 (socket says 172.21.15.64:34252) 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 -- 172.21.15.64:0/4084282207 learned_addr learned my addr 172.21.15.64:0/4084282207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c6ffd640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c77fe640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 0x7fa8c8142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 -- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 msgr2=0x7fa8c815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 -- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 msgr2=0x7fa8c8142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 0x7fa8c8142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 -- 172.21.15.64:0/4084282207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c815e170 con 0x7fa8c81412e0 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.932+0000 7fa8c7fff640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8c81412e0 0x7fa8c81441b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009e80 tx=0x7fa8c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.933+0000 7fa8bffff640 1 -- 172.21.15.64:0/4084282207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0044020 con 0x7fa8c81412e0 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.933+0000 7fa8bffff640 1 -- 172.21.15.64:0/4084282207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c003f470 con 0x7fa8c81412e0 2024-03-07T07:19:24.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.933+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c815de90 con 0x7fa8c81412e0 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.933+0000 7fa8bffff640 1 -- 172.21.15.64:0/4084282207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0039040 con 0x7fa8c81412e0 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.933+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c8141a40 con 0x7fa8c81412e0 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.934+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c80b35c0 con 0x7fa8c81412e0 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.934+0000 7fa8bffff640 1 -- 172.21.15.64:0/4084282207 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa8c0047020 con 0x7fa8c81412e0 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.934+0000 7fa8bffff640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa8a806b4d0 0x7fa8a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.934+0000 7fa8bffff640 1 -- 172.21.15.64:0/4084282207 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8c00acb30 con 0x7fa8c81412e0 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.934+0000 7fa8c77fe640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa8a806b4d0 0x7fa8a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:24.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.935+0000 7fa8c77fe640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa8a806b4d0 0x7fa8a806d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009ab0 tx=0x7fa8b0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:24.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:24.939+0000 7fa8bffff640 1 -- 172.21.15.64:0/4084282207 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa8c004e030 con 0x7fa8c81412e0 2024-03-07T07:19:25.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.121+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c8067240 con 0x7fa8c81412e0 2024-03-07T07:19:25.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.123+0000 7fa8bffff640 1 -- 172.21.15.64:0/4084282207 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fa8c007b3f0 con 0x7fa8c81412e0 2024-03-07T07:19:25.123 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:25.123 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa8a806b4d0 msgr2=0x7fa8a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa8a806b4d0 0x7fa8a806d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009ab0 tx=0x7fa8b0009240 comp rx=0 tx=0).stop 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8c81412e0 msgr2=0x7fa8c81441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8c81412e0 0x7fa8c81441b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009e80 tx=0x7fa8c0004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 shutdown_connections 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa8a806b4d0 0x7fa8a806d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8c81412e0 0x7fa8c81441b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8c80b2ca0 0x7fa8c815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 --2- 172.21.15.64:0/4084282207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8c8070940 0x7fa8c8142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 >> 172.21.15.64:0/4084282207 conn(0x7fa8c8072490 msgr2=0x7fa8c80a9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 shutdown_connections 2024-03-07T07:19:25.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.125+0000 7fa8cdfa7640 1 -- 172.21.15.64:0/4084282207 wait complete. 2024-03-07T07:19:25.136 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-07T07:19:25.136 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.237+0000 7fc36f989640 1 Processor -- start 2024-03-07T07:19:25.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.237+0000 7fc36f989640 1 -- start start 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36f989640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc3680d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36f989640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc368073020 0x7fc3680d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36f989640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36815fc30 con 0x7fc3680b5090 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36f989640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc368161ce0 con 0x7fc368073020 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36f989640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36815ff10 con 0x7fc368147bb0 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36df26640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc368073020 0x7fc3680d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36df26640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc368073020 0x7fc3680d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34262/0 (socket says 172.21.15.64:34262) 2024-03-07T07:19:25.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36df26640 1 -- 172.21.15.64:0/2251446567 learned_addr learned my addr 172.21.15.64:0/2251446567 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36d725640 1 --2- 172.21.15.64:0/2251446567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc3680d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36df26640 1 -- 172.21.15.64:0/2251446567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 msgr2=0x7fc3680d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36df26640 1 --2- 172.21.15.64:0/2251446567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc3680d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36df26640 1 -- 172.21.15.64:0/2251446567 >> v1:172.21.15.120:6789/0 conn(0x7fc3680b5090 legacy=0x7fc368072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc35ffff640 1 -- 172.21.15.64:0/2251446567 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537837541 0 0) 0x7fc36815fc30 con 0x7fc3680b5090 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36df26640 1 -- 172.21.15.64:0/2251446567 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3681601f0 con 0x7fc368073020 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.238+0000 7fc36d725640 1 --2- 172.21.15.64:0/2251446567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc3680d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.239+0000 7fc36df26640 1 --2- 172.21.15.64:0/2251446567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3680d1090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc364009e50 tx=0x7fc36402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f632a40d14ae23b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.239+0000 7fc35ffff640 1 -- 172.21.15.64:0/2251446567 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36402ca70 con 0x7fc368073020 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.239+0000 7fc35ffff640 1 -- 172.21.15.64:0/2251446567 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36402cc10 con 0x7fc368073020 2024-03-07T07:19:25.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.239+0000 7fc35ffff640 1 -- 172.21.15.64:0/2251446567 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640343f0 con 0x7fc368073020 2024-03-07T07:19:25.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 -- 172.21.15.64:0/2251446567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 msgr2=0x7fc3680d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 --2- 172.21.15.64:0/2251446567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3680d1090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc364009e50 tx=0x7fc36402bed0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 -- 172.21.15.64:0/2251446567 shutdown_connections 2024-03-07T07:19:25.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 --2- 172.21.15.64:0/2251446567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3680d1090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 --2- 172.21.15.64:0/2251446567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc3680d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 -- 172.21.15.64:0/2251446567 >> 172.21.15.64:0/2251446567 conn(0x7fc36806b480 msgr2=0x7fc36806b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 -- 172.21.15.64:0/2251446567 shutdown_connections 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.240+0000 7fc36f989640 1 -- 172.21.15.64:0/2251446567 wait complete. 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.241+0000 7fc36f989640 1 Processor -- start 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.241+0000 7fc36f989640 1 -- start start 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36f989640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3681f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36f989640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3680b5090 0x7fc3681437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36f989640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc36813b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36f989640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368147f90 con 0x7fc368073020 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36f989640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681607b0 con 0x7fc3680b5090 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36f989640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc368160cd0 con 0x7fc368147bb0 2024-03-07T07:19:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3681f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3681f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34264/0 (socket says 172.21.15.64:34264) 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 -- 172.21.15.64:0/2473312374 learned_addr learned my addr 172.21.15.64:0/2473312374 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36cf24640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3680b5090 0x7fc3681437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36df26640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc36813b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 -- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 msgr2=0x7fc36813b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc36813b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 -- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3680b5090 msgr2=0x7fc3681437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3680b5090 0x7fc3681437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 -- 172.21.15.64:0/2473312374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3681409f0 con 0x7fc368073020 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36cf24640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3680b5090 0x7fc3681437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36d725640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3681f81a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc36000e960 tx=0x7fc36000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc35dffb640 1 -- 172.21.15.64:0/2473312374 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360017070 con 0x7fc368073020 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc35dffb640 1 -- 172.21.15.64:0/2473312374 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000ca00 con 0x7fc368073020 2024-03-07T07:19:25.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368143f30 con 0x7fc368073020 2024-03-07T07:19:25.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.242+0000 7fc35dffb640 1 -- 172.21.15.64:0/2473312374 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000cdd0 con 0x7fc368073020 2024-03-07T07:19:25.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.243+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681444f0 con 0x7fc368073020 2024-03-07T07:19:25.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.244+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3680b76f0 con 0x7fc368073020 2024-03-07T07:19:25.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.244+0000 7fc35dffb640 1 -- 172.21.15.64:0/2473312374 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc36001a070 con 0x7fc368073020 2024-03-07T07:19:25.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.244+0000 7fc35dffb640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc34c06b4d0 0x7fc34c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.245+0000 7fc35dffb640 1 -- 172.21.15.64:0/2473312374 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc36008a030 con 0x7fc368073020 2024-03-07T07:19:25.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.245+0000 7fc36cf24640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc34c06b4d0 0x7fc34c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.245+0000 7fc36cf24640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc34c06b4d0 0x7fc34c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc358007fd0 tx=0x7fc358009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.249+0000 7fc35dffb640 1 -- 172.21.15.64:0/2473312374 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc3680b76f0 con 0x7fc368073020 2024-03-07T07:19:25.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.431+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc368062c90 con 0x7fc368073020 2024-03-07T07:19:25.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.432+0000 7fc35dffb640 1 -- 172.21.15.64:0/2473312374 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fc360058860 con 0x7fc368073020 2024-03-07T07:19:25.433 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:25.433 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:25.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc34c06b4d0 msgr2=0x7fc34c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc34c06b4d0 0x7fc34c06d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc358007fd0 tx=0x7fc358009040 comp rx=0 tx=0).stop 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 msgr2=0x7fc3681f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3681f81a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc36000e960 tx=0x7fc36000ee20 comp rx=0 tx=0).stop 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 shutdown_connections 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc34c06b4d0 0x7fc34c06d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc368147bb0 0x7fc36813b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3680b5090 0x7fc3681437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 --2- 172.21.15.64:0/2473312374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc368073020 0x7fc3681f81a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 >> 172.21.15.64:0/2473312374 conn(0x7fc36806b480 msgr2=0x7fc3680d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 shutdown_connections 2024-03-07T07:19:25.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.434+0000 7fc36f989640 1 -- 172.21.15.64:0/2473312374 wait complete. 2024-03-07T07:19:25.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-07T07:19:25.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-07T07:19:25.446 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-07T07:19:25.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-07T07:19:25.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-07T07:19:25.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-07T07:19:25.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-07T07:19:25.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-07T07:19:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.617+0000 7fdf7c7c9640 1 Processor -- start 2024-03-07T07:19:25.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.617+0000 7fdf7c7c9640 1 -- start start 2024-03-07T07:19:25.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.618+0000 7fdf7c7c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdf740cfa40 0x7fdf7406e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.618+0000 7fdf7c7c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7416e870 con 0x7fdf740b3c10 2024-03-07T07:19:25.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.618+0000 7fdf7c7c9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7416bc20 con 0x7fdf74071610 2024-03-07T07:19:25.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.618+0000 7fdf7c7c9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf74072420 con 0x7fdf740cfa40 2024-03-07T07:19:25.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fdf7a565640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fdf74071610 0x7fdf7416b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:47764/0 (socket says 172.21.15.64:47764) 2024-03-07T07:19:25.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fdf7a565640 1 -- 172.21.15.64:0/248290871 learned_addr learned my addr 172.21.15.64:0/248290871 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:25.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fdf7ad66640 1 --2- 172.21.15.64:0/248290871 >> v2:172.21.15.120:3300/0 conn(0x7fdf740cfa40 0x7fdf7406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7ff4b6dab640 1 Processor -- start 2024-03-07T07:19:25.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fad9686b640 1 Processor -- start 2024-03-07T07:19:25.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fad9686b640 1 -- start start 2024-03-07T07:19:25.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550087445 0 0) 0x7fdf7416e870 con 0x7fdf740b3c10 2024-03-07T07:19:25.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf5c005730 con 0x7fdf740b3c10 2024-03-07T07:19:25.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7ff4b6dab640 1 -- start start 2024-03-07T07:19:25.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.619+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715226075 0 0) 0x7fdf7416bc20 con 0x7fdf74071610 2024-03-07T07:19:25.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf7416e870 con 0x7fdf74071610 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737047690 0 0) 0x7fdf5c005730 con 0x7fdf740b3c10 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 >> v2:172.21.15.120:3300/0 conn(0x7fdf740cfa40 msgr2=0x7fdf7406e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 --2- 172.21.15.64:0/248290871 >> v2:172.21.15.120:3300/0 conn(0x7fdf740cfa40 0x7fdf7406e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 >> v1:172.21.15.120:6789/0 conn(0x7fdf74071610 legacy=0x7fdf7416b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74073040 con 0x7fdf740b3c10 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7fdf680015f0 con 0x7fdf740b3c10 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fad9686b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad90165ed0 0x7fad9006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fad9686b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad90071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fad9686b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad90071b50 0x7fad9006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fad9686b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad900734a0 con 0x7fad90165ed0 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fad9686b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad90071fe0 con 0x7fad90071b50 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fad9686b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad900ce540 con 0x7fad900b3c10 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad90071b50 0x7fad9006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad90071b50 0x7fad9006d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34116/0 (socket says 172.21.15.64:34116) 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 -- 172.21.15.64:0/3247828890 learned_addr learned my addr 172.21.15.64:0/3247828890 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad8f7fe640 1 --2- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad90071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad8ffff640 1 --2- 172.21.15.64:0/3247828890 >> v2:172.21.15.64:3300/0 conn(0x7fad90165ed0 0x7fad9006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 >> v1:172.21.15.64:6789/0 conn(0x7fdf740b3c10 legacy=0x7fdf740719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7f06abc83640 1 Processor -- start 2024-03-07T07:19:25.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 -- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 msgr2=0x7fad90071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 --2- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad90071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 -- 172.21.15.64:0/3247828890 >> v2:172.21.15.64:3300/0 conn(0x7fad90165ed0 msgr2=0x7fad9006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 --2- 172.21.15.64:0/3247828890 >> v2:172.21.15.64:3300/0 conn(0x7fad90165ed0 0x7fad9006ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7f7c8359e640 1 Processor -- start 2024-03-07T07:19:25.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad94e08640 1 -- 172.21.15.64:0/3247828890 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9006f200 con 0x7fad90071b50 2024-03-07T07:19:25.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf5c005d50 0x7fdf5c006130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf5c006a10 0x7fdf5c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf5c0073d0 0x7fdf5c0147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74072420 con 0x7fdf5c005d50 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c00bbc0 con 0x7fdf5c006a10 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c00bd90 con 0x7fdf5c0073d0 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf7a565640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf5c005d50 0x7fdf5c006130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf79d64640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf5c006a10 0x7fdf5c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.620+0000 7fdf7ad66640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf5c0073d0 0x7fdf5c0147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fdf7a565640 1 -- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf5c0073d0 msgr2=0x7fdf5c0147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fdf7a565640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf5c0073d0 0x7fdf5c0147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fdf7a565640 1 -- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf5c006a10 msgr2=0x7fdf5c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fdf7a565640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf5c006a10 0x7fdf5c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fdf7a565640 1 -- 172.21.15.64:0/248290871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74073040 con 0x7fdf5c005d50 2024-03-07T07:19:25.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7f7c8359e640 1 -- start start 2024-03-07T07:19:25.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7ff4b6dab640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4a8048ad0 0x7ff4a8052680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7ff4b6dab640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 0x7ff4a80537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7f06abc83640 1 -- start start 2024-03-07T07:19:25.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7ff4b6dab640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a803a2f0 con 0x7ff4a80eac70 2024-03-07T07:19:25.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7ff4b6dab640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a8052bc0 con 0x7ff4a8048ad0 2024-03-07T07:19:25.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7fdf7ad66640 1 -- 172.21.15.64:0/248290871 reap_dead start 2024-03-07T07:19:25.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7ff4b6dab640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a803a0a0 con 0x7ff4a803b0b0 2024-03-07T07:19:25.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7ff4b55a8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4a8048ad0 0x7ff4a8052680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7fdf7a565640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf5c005d50 0x7fdf5c006130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf64007da0 tx=0x7fdf5c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdc731a44a2a727 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7ff4b55a8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff4a8048ad0 0x7ff4a8052680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34292/0 (socket says 172.21.15.64:34292) 2024-03-07T07:19:25.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad8ffff640 1 --2- 172.21.15.64:0/3247828890 >> v2:172.21.15.64:3300/0 conn(0x7fad90165ed0 0x7fad9006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:25.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.621+0000 7fad8f7fe640 1 --2- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad90071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:25.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7ff4b55a8640 1 -- 172.21.15.64:0/3529478896 learned_addr learned my addr 172.21.15.64:0/3529478896 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7fad94e08640 1 --2- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9006d2a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad90068850 tx=0x7fad8802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e06ec5218f0a2ee server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7ff4b55a8640 1 -- 172.21.15.64:0/3529478896 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 msgr2=0x7ff4a80537a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-07T07:19:25.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7ff4b55a8640 1 --2- 172.21.15.64:0/3529478896 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 0x7ff4a80537a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7ff4b55a8640 1 -- 172.21.15.64:0/3529478896 >> v1:172.21.15.120:6789/0 conn(0x7ff4a80eac70 legacy=0x7ff4a80483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7ff4b55a8640 1 -- 172.21.15.64:0/3529478896 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a803a5d0 con 0x7ff4a8048ad0 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64007970 con 0x7fdf5c005d50 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64007310 con 0x7fdf5c005d50 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.622+0000 7fdf79563640 1 -- 172.21.15.64:0/248290871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6402c7c0 con 0x7fdf5c005d50 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f06abc83640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f06a41625d0 0x7f06a40b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f06abc83640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a40721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f06abc83640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a40d21c0 con 0x7f06a41629b0 2024-03-07T07:19:25.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f06abc83640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40d2dd0 con 0x7f06a41625d0 2024-03-07T07:19:25.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f06abc83640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40d24a0 con 0x7f06a406bc20 2024-03-07T07:19:25.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7fccc65fc640 1 Processor -- start 2024-03-07T07:19:25.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7fccc65fc640 1 -- start start 2024-03-07T07:19:25.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f06a921e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f06a41629b0 0x7f06a406b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48672/0 (socket says 172.21.15.64:48672) 2024-03-07T07:19:25.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f06a921e640 1 -- 172.21.15.64:0/3566219615 learned_addr learned my addr 172.21.15.64:0/3566219615 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:25.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f7c8259c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7c7c0e30c0 0x7f7c7c057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48682/0 (socket says 172.21.15.64:48682) 2024-03-07T07:19:25.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f315337e640 1 Processor -- start 2024-03-07T07:19:25.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f7c8259c640 1 -- 172.21.15.64:0/806508095 learned_addr learned my addr 172.21.15.64:0/806508095 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:25.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7fdf7c7c9640 1 -- 172.21.15.64:0/248290871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf5c005d50 msgr2=0x7fdf5c006130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7fdf7c7c9640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf5c005d50 0x7fdf5c006130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf64007da0 tx=0x7fdf5c007a40 comp rx=0 tx=0).stop 2024-03-07T07:19:25.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7f315337e640 1 -- start start 2024-03-07T07:19:25.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fdf7c7c9640 1 -- 172.21.15.64:0/248290871 shutdown_connections 2024-03-07T07:19:25.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fdf7c7c9640 1 --2- 172.21.15.64:0/248290871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf5c005d50 0x7fdf5c006130 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf64007da0 tx=0x7fdf5c007a40 comp rx=0 tx=0).stop 2024-03-07T07:19:25.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fdf7c7c9640 1 -- 172.21.15.64:0/248290871 >> 172.21.15.64:0/248290871 conn(0x7fdf740b3120 msgr2=0x7fdf740b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c8359e640 1 --2- 172.21.15.64:0/806508095 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c03e840 0x7f7c7c053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c8359e640 1 --2- 172.21.15.64:0/806508095 >> v2:172.21.15.64:3300/0 conn(0x7f7c7c054130 0x7f7c7c0068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c81d9b640 1 --2- 172.21.15.64:0/806508095 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c03e840 0x7f7c7c053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7380816640 1 Processor -- start 2024-03-07T07:19:25.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7fad8effd640 1 -- 172.21.15.64:0/3247828890 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8802ca80 con 0x7fad90071b50 2024-03-07T07:19:25.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.623+0000 7fad8effd640 1 -- 172.21.15.64:0/3247828890 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8802cc20 con 0x7fad90071b50 2024-03-07T07:19:25.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fad9686b640 1 -- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 msgr2=0x7fad9006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fad9686b640 1 --2- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9006d2a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad90068850 tx=0x7fad8802bee0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7ff4b55a8640 1 --2- 172.21.15.64:0/3529478896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a8048ad0 0x7ff4a8052680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009870 tx=0x7ff4a002b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71d51b7cf4e44be server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7380816640 1 -- start start 2024-03-07T07:19:25.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fccc65fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc0144860 0x7fccc006ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f06a9a1f640 1 --2- 172.21.15.64:0/3566219615 >> v2:172.21.15.120:3300/0 conn(0x7f06a41625d0 0x7f06a40b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f06aa220640 1 --2- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a40721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c7c058570 con 0x7f7c7c0e30c0 2024-03-07T07:19:25.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7c058850 con 0x7f7c7c054130 2024-03-07T07:19:25.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c058b30 con 0x7f7c7c03e840 2024-03-07T07:19:25.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fccbffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc0144860 0x7fccc006ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c8159a640 1 -- 172.21.15.64:0/806508095 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980532860 0 0) 0x7f7c7c058570 con 0x7f7c7c0e30c0 2024-03-07T07:19:25.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c8159a640 1 -- 172.21.15.64:0/806508095 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6c003520 con 0x7f7c7c0e30c0 2024-03-07T07:19:25.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fccbffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc0144860 0x7fccc006ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:56860/0 (socket says 172.21.15.64:56860) 2024-03-07T07:19:25.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7fccbffff640 1 -- 172.21.15.64:0/1121091962 learned_addr learned my addr 172.21.15.64:0/1121091962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f06a8a1d640 1 -- 172.21.15.64:0/3566219615 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947276484 0 0) 0x7f06a40d21c0 con 0x7f06a41629b0 2024-03-07T07:19:25.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fdf7c7c9640 1 -- 172.21.15.64:0/248290871 shutdown_connections 2024-03-07T07:19:25.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccc65fc640 1 --2- 172.21.15.64:0/1121091962 >> v2:172.21.15.64:3300/0 conn(0x7fccc0071c60 0x7fccc00b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc00cd700 con 0x7fccc0144c40 2024-03-07T07:19:25.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc00ce310 con 0x7fccc0071c60 2024-03-07T07:19:25.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00cd9e0 con 0x7fccc0144860 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f06a9a1f640 1 -- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 msgr2=0x7f06a40721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f06a9a1f640 1 --2- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a40721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f06a9a1f640 1 -- 172.21.15.64:0/3566219615 >> v1:172.21.15.64:6789/0 conn(0x7f06a41629b0 legacy=0x7f06a406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f06a9a1f640 1 -- 172.21.15.64:0/3566219615 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40d2780 con 0x7f06a41625d0 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315337e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c141170 0x7f314c144450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315337e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f314c161e10 0x7f314c1419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315337e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c15e820 con 0x7f314c0b5610 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315337e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c144990 con 0x7f314c161e10 2024-03-07T07:19:25.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315337e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0b9810 con 0x7f314c141170 2024-03-07T07:19:25.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccc4b99640 1 --2- 172.21.15.64:0/1121091962 >> v2:172.21.15.64:3300/0 conn(0x7fccc0071c60 0x7fccc00b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7ff4b4da7640 1 -- 172.21.15.64:0/3529478896 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037070 con 0x7ff4a8048ad0 2024-03-07T07:19:25.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315111a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c141170 0x7f314c144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f3150919640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f314c161e10 0x7f314c1419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315111a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c141170 0x7f314c144450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:56872/0 (socket says 172.21.15.64:56872) 2024-03-07T07:19:25.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f315111a640 1 -- 172.21.15.64:0/2374464782 learned_addr learned my addr 172.21.15.64:0/2374464782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccbffff640 1 -- 172.21.15.64:0/1121091962 >> v2:172.21.15.64:3300/0 conn(0x7fccc0071c60 msgr2=0x7fccc00b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccbffff640 1 --2- 172.21.15.64:0/1121091962 >> v2:172.21.15.64:3300/0 conn(0x7fccc0071c60 0x7fccc00b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f3150919640 1 -- 172.21.15.64:0/2374464782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c141170 msgr2=0x7f314c144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f3150919640 1 --2- 172.21.15.64:0/2374464782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c141170 0x7f314c144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f3150919640 1 -- 172.21.15.64:0/2374464782 >> v1:172.21.15.120:6789/0 conn(0x7f314c0b5610 legacy=0x7f314c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f3150919640 1 -- 172.21.15.64:0/2374464782 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c162310 con 0x7f314c161e10 2024-03-07T07:19:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fccbffff640 1 -- 172.21.15.64:0/1121091962 >> v1:172.21.15.120:6789/0 conn(0x7fccc0144c40 legacy=0x7fccc0071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7ff4b4da7640 1 -- 172.21.15.64:0/3529478896 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0030710 con 0x7ff4a8048ad0 2024-03-07T07:19:25.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fccbffff640 1 -- 172.21.15.64:0/1121091962 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc00cdcc0 con 0x7fccc0144860 2024-03-07T07:19:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7380816640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f737c15bec0 0x7f737c160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b4da7640 1 -- 172.21.15.64:0/3529478896 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037070 con 0x7ff4a8048ad0 2024-03-07T07:19:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.624+0000 7f7c82d9d640 1 --2- 172.21.15.64:0/806508095 >> v2:172.21.15.64:3300/0 conn(0x7f7c7c054130 0x7f7c7c0068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f7c81d9b640 1 -- 172.21.15.64:0/806508095 >> v2:172.21.15.64:3300/0 conn(0x7f7c7c054130 msgr2=0x7f7c7c0068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f7c81d9b640 1 --2- 172.21.15.64:0/806508095 >> v2:172.21.15.64:3300/0 conn(0x7f7c7c054130 0x7f7c7c0068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f7c81d9b640 1 -- 172.21.15.64:0/806508095 >> v1:172.21.15.64:6789/0 conn(0x7f7c7c0e30c0 legacy=0x7f7c7c057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7f7c81d9b640 1 -- 172.21.15.64:0/806508095 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c0063b0 con 0x7f7c7c03e840 2024-03-07T07:19:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7380816640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 0x7f737c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 -- 172.21.15.64:0/3529478896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a8048ad0 msgr2=0x7ff4a8052680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 --2- 172.21.15.64:0/3529478896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a8048ad0 0x7ff4a8052680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009870 tx=0x7ff4a002b6a0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f06aa220640 1 --2- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a40721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 -- 172.21.15.64:0/3529478896 shutdown_connections 2024-03-07T07:19:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 --2- 172.21.15.64:0/3529478896 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 0x7ff4a80537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 --2- 172.21.15.64:0/3529478896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a8048ad0 0x7ff4a8052680 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009870 tx=0x7ff4a002b6a0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 -- 172.21.15.64:0/3529478896 >> 172.21.15.64:0/3529478896 conn(0x7ff4a801e6c0 msgr2=0x7ff4a80553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7380816640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f737c06cc30 0x7f737c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7380816640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c0b5930 con 0x7f737c06cc30 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7380816640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c15faa0 con 0x7f737c15bec0 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7380816640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c15fd80 con 0x7f737c0b72d0 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7379d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 0x7f737c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fad9686b640 1 -- 172.21.15.64:0/3247828890 shutdown_connections 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fad9686b640 1 --2- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9006d2a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fad9686b640 1 --2- 172.21.15.64:0/3247828890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad90071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fad9686b640 1 --2- 172.21.15.64:0/3247828890 >> v2:172.21.15.64:3300/0 conn(0x7fad90165ed0 0x7fad9006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fad9686b640 1 -- 172.21.15.64:0/3247828890 >> 172.21.15.64:0/3247828890 conn(0x7fad900b3120 msgr2=0x7fad900b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 -- 172.21.15.64:0/3529478896 shutdown_connections 2024-03-07T07:19:25.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f7c81d9b640 1 --2- 172.21.15.64:0/806508095 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c03e840 0x7f7c7c053bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c780097a0 tx=0x7f7c7802b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef8fd8dddba5d10 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f737a59c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f737c15bec0 0x7f737c160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f737ad9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f737c06cc30 0x7f737c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f3150919640 1 --2- 172.21.15.64:0/2374464782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c161e10 0x7f314c1419b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f313c0097a0 tx=0x7f313c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b38a27f19aba33 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f06a9a1f640 1 --2- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a41625d0 0x7f06a40b71d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06a00097a0 tx=0x7f06a002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc41651e61dcf4f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fad9686b640 1 -- 172.21.15.64:0/3247828890 shutdown_connections 2024-03-07T07:19:25.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7fad9686b640 1 -- 172.21.15.64:0/3247828890 wait complete. 2024-03-07T07:19:25.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f737ad9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f737c06cc30 0x7f737c06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34332/0 (socket says 172.21.15.64:34332) 2024-03-07T07:19:25.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f737a59c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f737c15bec0 0x7f737c160580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34148/0 (socket says 172.21.15.64:34148) 2024-03-07T07:19:25.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7f737ad9d640 1 -- 172.21.15.64:0/596687328 learned_addr learned my addr 172.21.15.64:0/596687328 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f3143fff640 1 -- 172.21.15.64:0/2374464782 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c035020 con 0x7f314c161e10 2024-03-07T07:19:25.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7fccc4b99640 1 --2- 172.21.15.64:0/1121091962 >> v2:172.21.15.64:3300/0 conn(0x7fccc0071c60 0x7fccc00b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:25.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f3143fff640 1 -- 172.21.15.64:0/2374464782 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c031810 con 0x7f314c161e10 2024-03-07T07:19:25.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f3143fff640 1 -- 172.21.15.64:0/2374464782 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c031be0 con 0x7f314c161e10 2024-03-07T07:19:25.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7fccbffff640 1 --2- 172.21.15.64:0/1121091962 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc0144860 0x7fccc006ecf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009e50 tx=0x7fccb802bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f07ab4124da14da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7fccbeffd640 1 -- 172.21.15.64:0/1121091962 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb802caa0 con 0x7fccc0144860 2024-03-07T07:19:25.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7fccbeffd640 1 -- 172.21.15.64:0/1121091962 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb802cc40 con 0x7fccc0144860 2024-03-07T07:19:25.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7fccbeffd640 1 -- 172.21.15.64:0/1121091962 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb80343f0 con 0x7fccc0144860 2024-03-07T07:19:25.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7ff4b6dab640 1 -- 172.21.15.64:0/3529478896 wait complete. 2024-03-07T07:19:25.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 -- 172.21.15.64:0/2374464782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c161e10 msgr2=0x7f314c1419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 --2- 172.21.15.64:0/2374464782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c161e10 0x7f314c1419b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f313c0097a0 tx=0x7f313c02b8b0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 -- 172.21.15.64:0/2374464782 shutdown_connections 2024-03-07T07:19:25.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 --2- 172.21.15.64:0/2374464782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c161e10 0x7f314c1419b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 --2- 172.21.15.64:0/2374464782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c141170 0x7f314c144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 -- 172.21.15.64:0/2374464782 >> 172.21.15.64:0/2374464782 conn(0x7f314c0d2270 msgr2=0x7f314c0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f737ad9d640 1 -- 172.21.15.64:0/596687328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 msgr2=0x7f737c0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f737ad9d640 1 --2- 172.21.15.64:0/596687328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 0x7f737c0b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f737ad9d640 1 -- 172.21.15.64:0/596687328 >> v2:172.21.15.120:3300/0 conn(0x7f737c15bec0 msgr2=0x7f737c160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f737ad9d640 1 --2- 172.21.15.64:0/596687328 >> v2:172.21.15.120:3300/0 conn(0x7f737c15bec0 0x7f737c160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 -- 172.21.15.64:0/2374464782 shutdown_connections 2024-03-07T07:19:25.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f737ad9d640 1 -- 172.21.15.64:0/596687328 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c160060 con 0x7f737c06cc30 2024-03-07T07:19:25.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7ff4b6dab640 1 Processor -- start 2024-03-07T07:19:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.625+0000 7fdf7c7c9640 1 -- 172.21.15.64:0/248290871 wait complete. 2024-03-07T07:19:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc0144860 msgr2=0x7fccc006ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 --2- 172.21.15.64:0/1121091962 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc0144860 0x7fccc006ecf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009e50 tx=0x7fccb802bf00 comp rx=0 tx=0).stop 2024-03-07T07:19:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 shutdown_connections 2024-03-07T07:19:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 --2- 172.21.15.64:0/1121091962 >> v2:172.21.15.64:3300/0 conn(0x7fccc0071c60 0x7fccc00b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 --2- 172.21.15.64:0/1121091962 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc0144860 0x7fccc006ecf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 >> 172.21.15.64:0/1121091962 conn(0x7fccc006b040 msgr2=0x7fccc006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f315337e640 1 -- 172.21.15.64:0/2374464782 wait complete. 2024-03-07T07:19:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 shutdown_connections 2024-03-07T07:19:25.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fccc65fc640 1 -- 172.21.15.64:0/1121091962 wait complete. 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f06a8a1d640 1 -- 172.21.15.64:0/3566219615 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035020 con 0x7f06a41625d0 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f06a8a1d640 1 -- 172.21.15.64:0/3566219615 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0031640 con 0x7f06a41625d0 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.627+0000 7f06a8a1d640 1 -- 172.21.15.64:0/3566219615 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035020 con 0x7f06a41625d0 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f06abc83640 1 -- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a41625d0 msgr2=0x7f06a40b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f06abc83640 1 --2- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a41625d0 0x7f06a40b71d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06a00097a0 tx=0x7f06a002b8f0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f06abc83640 1 -- 172.21.15.64:0/3566219615 shutdown_connections 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f06abc83640 1 --2- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a40721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f06abc83640 1 --2- 172.21.15.64:0/3566219615 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a41625d0 0x7f06a40b71d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7f06abc83640 1 -- 172.21.15.64:0/3566219615 >> 172.21.15.64:0/3566219615 conn(0x7f06a406d4c0 msgr2=0x7f06a4066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fad9686b640 1 Processor -- start 2024-03-07T07:19:25.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7f06abc83640 1 -- 172.21.15.64:0/3566219615 shutdown_connections 2024-03-07T07:19:25.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7f06abc83640 1 -- 172.21.15.64:0/3566219615 wait complete. 2024-03-07T07:19:25.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7f737ad9d640 1 --2- 172.21.15.64:0/596687328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c06d0c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f737c0d1ff0 tx=0x7f7374030260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56daf66527628988 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7ff4b6dab640 1 -- start start 2024-03-07T07:19:25.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7ff4b6dab640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 0x7ff4a80e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f315337e640 1 Processor -- start 2024-03-07T07:19:25.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7ff4b6dab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80eac70 0x7ff4a8031fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7ff4b6dab640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4a804bc50 0x7ff4a8185b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7ff4b6dab640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80eb050 con 0x7ff4a80eac70 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7ff4b6dab640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a803ac20 con 0x7ff4a804bc50 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7ff4b6dab640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80eb1d0 con 0x7ff4a803b0b0 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7ff4b65aa640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4a804bc50 0x7ff4a8185b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7ff4b55a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80eac70 0x7ff4a8031fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7ff4b55a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80eac70 0x7ff4a8031fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34350/0 (socket says 172.21.15.64:34350) 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7ff4b55a8640 1 -- 172.21.15.64:0/2369561203 learned_addr learned my addr 172.21.15.64:0/2369561203 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7f7c8159a640 1 -- 172.21.15.64:0/806508095 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78037070 con 0x7f7c7c03e840 2024-03-07T07:19:25.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7f7c8159a640 1 -- 172.21.15.64:0/806508095 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78030510 con 0x7f7c7c03e840 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7f7c8159a640 1 -- 172.21.15.64:0/806508095 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74001670 con 0x7f7c7c03e840 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c03e840 msgr2=0x7f7c7c053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f7c8359e640 1 --2- 172.21.15.64:0/806508095 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c03e840 0x7f7c7c053bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c780097a0 tx=0x7f7c7802b760 comp rx=0 tx=0).stop 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fad9686b640 1 -- start start 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad9686b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad9686b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad900c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad9686b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad90165ed0 0x7fad90163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad9686b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad90072270 con 0x7fad90165ed0 2024-03-07T07:19:25.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad9686b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad900ce260 con 0x7fad90071b50 2024-03-07T07:19:25.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad9686b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad90072550 con 0x7fad900b3c10 2024-03-07T07:19:25.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9014f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34174/0 (socket says 172.21.15.64:34174) 2024-03-07T07:19:25.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fccc65fc640 1 Processor -- start 2024-03-07T07:19:25.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 -- 172.21.15.64:0/3331023280 learned_addr learned my addr 172.21.15.64:0/3331023280 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8f7fe640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad900c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f315337e640 1 -- start start 2024-03-07T07:19:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 shutdown_connections 2024-03-07T07:19:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f7c8359e640 1 --2- 172.21.15.64:0/806508095 >> v2:172.21.15.64:3300/0 conn(0x7f7c7c054130 0x7f7c7c0068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f7c8359e640 1 --2- 172.21.15.64:0/806508095 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c03e840 0x7f7c7c053bf0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c780097a0 tx=0x7f7c7802b760 comp rx=0 tx=0).stop 2024-03-07T07:19:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 >> 172.21.15.64:0/806508095 conn(0x7f7c7c0559e0 msgr2=0x7f7c7c055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 -- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 msgr2=0x7fad900c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad900b3c10 0x7fad900c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 -- 172.21.15.64:0/3331023280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad90165ed0 msgr2=0x7fad90163070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:19:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f737959a640 1 -- 172.21.15.64:0/596687328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374030e00 con 0x7f737c06cc30 2024-03-07T07:19:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad94e08640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad90165ed0 0x7fad90163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad90165ed0 0x7fad90163070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fad8ffff640 1 -- 172.21.15.64:0/3331023280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad900acea0 con 0x7fad90071b50 2024-03-07T07:19:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f737959a640 1 -- 172.21.15.64:0/596687328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73740383f0 con 0x7f737c06cc30 2024-03-07T07:19:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7f737959a640 1 -- 172.21.15.64:0/596687328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740387c0 con 0x7f737c06cc30 2024-03-07T07:19:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7ff4b5da9640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 0x7ff4a80e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7ff4b55a8640 1 -- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 msgr2=0x7ff4a80e6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7ff4b55a8640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4a803b0b0 0x7ff4a80e6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7ff4b55a8640 1 -- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4a804bc50 msgr2=0x7ff4a8185b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7ff4b55a8640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4a804bc50 0x7ff4a8185b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7ff4b55a8640 1 -- 172.21.15.64:0/2369561203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a8050bb0 con 0x7ff4a80eac70 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fdf7c7c9640 1 Processor -- start 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.626+0000 7fdf7c7c9640 1 -- start start 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7c7c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf741599d0 0x7fdf740ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7c7c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf74159db0 0x7fdf740afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7a565640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf741599d0 0x7fdf740ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7a565640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf741599d0 0x7fdf740ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34348/0 (socket says 172.21.15.64:34348) 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7c7c9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf740b43d0 0x7fdf740b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7c7c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf740ce3a0 con 0x7fdf741599d0 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7c7c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7406e130 con 0x7fdf74159db0 2024-03-07T07:19:25.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7c7c9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf740ce520 con 0x7fdf740b43d0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7a565640 1 -- 172.21.15.64:0/3917944578 learned_addr learned my addr 172.21.15.64:0/3917944578 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf7ad66640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf740b43d0 0x7fdf740b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.628+0000 7fdf79d64640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf74159db0 0x7fdf740afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf7a565640 1 -- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf740b43d0 msgr2=0x7fdf740b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf7a565640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf740b43d0 0x7fdf740b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf7a565640 1 -- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf74159db0 msgr2=0x7fdf740afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf7a565640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf74159db0 0x7fdf740afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf7a565640 1 -- 172.21.15.64:0/3917944578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf740b0560 con 0x7fdf741599d0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf7ad66640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdf740b43d0 0x7fdf740b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf7a565640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf741599d0 0x7fdf740ae3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf6402c230 tx=0x7fdf64004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64044070 con 0x7fdf741599d0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.629+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64005690 con 0x7fdf741599d0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64005a40 con 0x7fdf741599d0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fdf7c7c9640 1 -- 172.21.15.64:0/3917944578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf740b0840 con 0x7fdf741599d0 2024-03-07T07:19:25.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.630+0000 7fdf7c7c9640 1 -- 172.21.15.64:0/3917944578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf740b0e00 con 0x7fdf741599d0 2024-03-07T07:19:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f06abc83640 1 Processor -- start 2024-03-07T07:19:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f06abc83640 1 -- start start 2024-03-07T07:19:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7fccc65fc640 1 -- start start 2024-03-07T07:19:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f315337e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c0b5610 0x7f314c160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f315337e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c141170 0x7f314c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f315337e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c161e10 0x7f314c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f315337e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c162bb0 con 0x7f314c141170 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f315337e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c1628d0 con 0x7f314c0b5610 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f315337e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c141ef0 con 0x7f314c161e10 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7f7380816640 1 -- 172.21.15.64:0/596687328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 msgr2=0x7f737c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7380816640 1 --2- 172.21.15.64:0/596687328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c06d0c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f737c0d1ff0 tx=0x7f7374030260 comp rx=0 tx=0).stop 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7fad8ffff640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9014f550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad8400d800 tx=0x7fad8400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84018070 con 0x7fad90071b50 2024-03-07T07:19:25.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad84004030 con 0x7fad90071b50 2024-03-07T07:19:25.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.631+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84018070 con 0x7fad90071b50 2024-03-07T07:19:25.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f315191b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c161e10 0x7f314c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f315191b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c161e10 0x7f314c156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:56910/0 (socket says 172.21.15.64:56910) 2024-03-07T07:19:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7380816640 1 -- 172.21.15.64:0/596687328 shutdown_connections 2024-03-07T07:19:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7380816640 1 --2- 172.21.15.64:0/596687328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c06d0c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7380816640 1 --2- 172.21.15.64:0/596687328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 0x7f737c0b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7380816640 1 --2- 172.21.15.64:0/596687328 >> v2:172.21.15.120:3300/0 conn(0x7f737c15bec0 0x7f737c160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7380816640 1 -- 172.21.15.64:0/596687328 >> 172.21.15.64:0/596687328 conn(0x7f737c070400 msgr2=0x7f737c070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccc65fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc0071c60 0x7fccc01f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad900ad180 con 0x7fad90071b50 2024-03-07T07:19:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad900ad740 con 0x7fad90071b50 2024-03-07T07:19:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f315191b640 1 -- 172.21.15.64:0/284882379 learned_addr learned my addr 172.21.15.64:0/284882379 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f315111a640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c0b5610 0x7f314c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f3150919640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c141170 0x7f314c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf74062c90 con 0x7fdf741599d0 2024-03-07T07:19:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 shutdown_connections 2024-03-07T07:19:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f315191b640 1 -- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c0b5610 msgr2=0x7f314c160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f315191b640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c0b5610 0x7f314c160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f315191b640 1 -- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c141170 msgr2=0x7f314c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f315191b640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c141170 0x7f314c06b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7ff4b55a8640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80eac70 0x7ff4a8031fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0035040 tx=0x7ff4a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f315191b640 1 -- 172.21.15.64:0/284882379 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c06b580 con 0x7f314c161e10 2024-03-07T07:19:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f7380816640 1 -- 172.21.15.64:0/596687328 shutdown_connections 2024-03-07T07:19:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdf6403d070 con 0x7fdf741599d0 2024-03-07T07:19:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f315111a640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c0b5610 0x7f314c160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f3150919640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f314c141170 0x7f314c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0041020 con 0x7ff4a80eac70 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccc65fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccc0144860 0x7fccc01f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccc65fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc00cf600 0x7fccc0158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccc65fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00ce7a0 con 0x7fccc0071c60 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccc65fc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc006e600 con 0x7fccc0144860 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccc65fc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc01519d0 con 0x7fccc00cf600 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccbf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccc0144860 0x7fccc01f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccbf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccc0144860 0x7fccc01f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34202/0 (socket says 172.21.15.64:34202) 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccbf7fe640 1 -- 172.21.15.64:0/1815940966 learned_addr learned my addr 172.21.15.64:0/1815940966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccc4b99640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc00cf600 0x7fccc0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7fccbffff640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc0071c60 0x7fccc01f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbf7fe640 1 -- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc00cf600 msgr2=0x7fccc0158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbf7fe640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccc00cf600 0x7fccc0158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbf7fe640 1 -- 172.21.15.64:0/1815940966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc0071c60 msgr2=0x7fccc01f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbf7fe640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc0071c60 0x7fccc01f8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbf7fe640 1 -- 172.21.15.64:0/1815940966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0143aa0 con 0x7fccc0144860 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7ff4b6dab640 1 -- 172.21.15.64:0/2369561203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a8050e90 con 0x7ff4a80eac70 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7ff4b6dab640 1 -- 172.21.15.64:0/2369561203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a8051450 con 0x7ff4a80eac70 2024-03-07T07:19:25.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0003610 con 0x7ff4a80eac70 2024-03-07T07:19:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.632+0000 7f7c8359e640 1 -- 172.21.15.64:0/806508095 wait complete. 2024-03-07T07:19:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f315191b640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c161e10 0x7f314c156580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f314400c730 tx=0x7f314400cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0037040 con 0x7ff4a80eac70 2024-03-07T07:19:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fdf72ffd640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fdf6006b5d0 0x7fdf6006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf640ab4d0 con 0x7fdf741599d0 2024-03-07T07:19:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fdf72ffd640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7fdf60073770 0x7fdf60075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 --> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf64007310 con 0x7fdf60073770 2024-03-07T07:19:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf640b2050 con 0x7fdf741599d0 2024-03-07T07:19:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f7380816640 1 -- 172.21.15.64:0/596687328 wait complete. 2024-03-07T07:19:25.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f06abc83640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a41637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7fdf7ad66640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7fdf60073770 0x7fdf60075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f06abc83640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41625d0 0x7f06a4153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f06abc83640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a41629b0 0x7f06a4164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f06abc83640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40d1e70 con 0x7f06a41625d0 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f06abc83640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40cea40 con 0x7f06a41629b0 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7f06abc83640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40b15d0 con 0x7f06a406bc20 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a41637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a41637a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:56934/0 (socket says 172.21.15.64:56934) 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 -- 172.21.15.64:0/1881716492 learned_addr learned my addr 172.21.15.64:0/1881716492 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 -- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a41629b0 msgr2=0x7f06a4164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06a41629b0 0x7f06a4164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 -- 172.21.15.64:0/1881716492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41625d0 msgr2=0x7f06a4153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06a41625d0 0x7f06a4153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 -- 172.21.15.64:0/1881716492 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a415a5c0 con 0x7f06a406bc20 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7fdf7ad66640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7fdf60073770 0x7fdf60075b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f7c8359e640 1 Processor -- start 2024-03-07T07:19:25.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7fdf79d64640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fdf6006b5d0 0x7fdf6006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7ff4b6dab640 1 -- 172.21.15.64:0/2369561203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff4a8051830 con 0x7ff4a80eac70 2024-03-07T07:19:25.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7f06a9a1f640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a41637a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f7c8359e640 1 -- start start 2024-03-07T07:19:25.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fad840041e0 con 0x7fad90071b50 2024-03-07T07:19:25.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.634+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad90062c90 con 0x7fad90071b50 2024-03-07T07:19:25.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035040 con 0x7f06a406bc20 2024-03-07T07:19:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fad8cff9640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7c06b6d0 0x7fad7c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad8408a170 con 0x7fad90071b50 2024-03-07T07:19:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0003820 con 0x7f06a406bc20 2024-03-07T07:19:25.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144016020 con 0x7f314c161e10 2024-03-07T07:19:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f7380816640 1 Processor -- start 2024-03-07T07:19:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fad8cff9640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fad7c073870 0x7fad7c075c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f7380816640 1 -- start start 2024-03-07T07:19:25.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3144006d50 con 0x7f314c161e10 2024-03-07T07:19:25.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144003bb0 con 0x7f314c161e10 2024-03-07T07:19:25.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 --> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8401ee50 con 0x7fad7c073870 2024-03-07T07:19:25.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad84091050 con 0x7fad90071b50 2024-03-07T07:19:25.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fad8f7fe640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7c06b6d0 0x7fad7c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== osd.3 v2:172.21.15.64:6824/2270479097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdf64007310 con 0x7fdf60073770 2024-03-07T07:19:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.636+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff4a0044020 con 0x7ff4a80eac70 2024-03-07T07:19:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbf7fe640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccc0144860 0x7fccc01f87c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccb000b3f0 tx=0x7fccb000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.636+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a002d040 con 0x7f06a406bc20 2024-03-07T07:19:25.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.636+0000 7fdf79d64640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fdf6006b5d0 0x7fdf6006da90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf68001760 tx=0x7fdf68000f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.636+0000 7f06abc83640 1 -- 172.21.15.64:0/1881716492 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a4163dc0 con 0x7f06a406bc20 2024-03-07T07:19:25.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.636+0000 7fad8f7fe640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7c06b6d0 0x7fad7c06db90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad80009560 tx=0x7fad80009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.636+0000 7fad94e08640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fad7c073870 0x7fad7c075c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7ff4a67fc640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff49406b520 0x7ff49406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f06abc83640 1 -- 172.21.15.64:0/1881716492 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a4164380 con 0x7f06a406bc20 2024-03-07T07:19:25.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0015020 con 0x7fccc0144860 2024-03-07T07:19:25.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb0011830 con 0x7fccc0144860 2024-03-07T07:19:25.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.633+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0011c20 con 0x7fccc0144860 2024-03-07T07:19:25.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7380816640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c14dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fccc65fc640 1 -- 172.21.15.64:0/1815940966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc0143d80 con 0x7fccc0144860 2024-03-07T07:19:25.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.635+0000 7fccc65fc640 1 -- 172.21.15.64:0/1815940966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc0156900 con 0x7fccc0144860 2024-03-07T07:19:25.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8359e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c054130 0x7f7c7c0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c7c0e30c0 0x7f7c7c0de300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4a00abb30 con 0x7ff4a80eac70 2024-03-07T07:19:25.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fccb0018020 con 0x7fccc0144860 2024-03-07T07:19:25.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8359e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c7c0d2130 0x7f7c7c0de840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c006e10 con 0x7f7c7c0e30c0 2024-03-07T07:19:25.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8359e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0e34a0 con 0x7f7c7c0d2130 2024-03-07T07:19:25.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8359e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c006f90 con 0x7f7c7c054130 2024-03-07T07:19:25.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7ff4a67fc640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7ff4940736c0 0x7ff494075ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7ff4b5da9640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff49406b520 0x7ff49406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 --> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4a0033e50 con 0x7ff4940736c0 2024-03-07T07:19:25.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4a0076780 con 0x7ff4a80eac70 2024-03-07T07:19:25.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f315337e640 1 -- 172.21.15.64:0/284882379 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c06b850 con 0x7f314c161e10 2024-03-07T07:19:25.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f315337e640 1 -- 172.21.15.64:0/284882379 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c06be10 con 0x7f314c161e10 2024-03-07T07:19:25.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7fccc65fc640 1 -- 172.21.15.64:0/1815940966 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fccc0156470 con 0x7fccc0144860 2024-03-07T07:19:25.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8259c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c054130 0x7f7c7c0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c82d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c7c0d2130 0x7f7c7c0de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8259c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c054130 0x7f7c7c0cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:56944/0 (socket says 172.21.15.64:56944) 2024-03-07T07:19:25.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7fccbcff9640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fccac06b5a0 0x7fccac06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c8259c640 1 -- 172.21.15.64:0/3096124966 learned_addr learned my addr 172.21.15.64:0/3096124966 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7c81d9b640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c7c0e30c0 0x7f7c7c0de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7fad94e08640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fad7c073870 0x7fad7c075c90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccb00899c0 con 0x7fccc0144860 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7fccbcff9640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7fccac073820 0x7fccac075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 --> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccb000ce50 con 0x7fccac073820 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccb0090050 con 0x7fccc0144860 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7380816640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 0x7f737c15e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7380816640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c15bec0 0x7f737c152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7380816640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0b5680 con 0x7f737c06cc30 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7380816640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0b5500 con 0x7f737c15bec0 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7f7380816640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0cead0 con 0x7f737c0b72d0 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c14dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c14dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34398/0 (socket says 172.21.15.64:34398) 2024-03-07T07:19:25.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 -- 172.21.15.64:0/1587220573 learned_addr learned my addr 172.21.15.64:0/1587220573 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7fccbffff640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fccac06b5a0 0x7fccac06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737ad9d640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c15bec0 0x7f737c152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f06abc83640 1 -- 172.21.15.64:0/1881716492 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f06a4153560 con 0x7f06a406bc20 2024-03-07T07:19:25.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f7c8259c640 1 -- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c7c0d2130 msgr2=0x7f7c7c0de840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f7c8259c640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c7c0d2130 0x7f7c7c0de840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f7c8259c640 1 -- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c7c0e30c0 msgr2=0x7f7c7c0de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f7c8259c640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c7c0e30c0 0x7f7c7c0de300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f7c8259c640 1 -- 172.21.15.64:0/3096124966 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c0ded80 con 0x7f7c7c054130 2024-03-07T07:19:25.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 -- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 msgr2=0x7f737c15e280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T07:19:25.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f737c0b72d0 0x7f737c15e280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 -- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c15bec0 msgr2=0x7f737c152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c15bec0 0x7f737c152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.637+0000 7ff4b65aa640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7ff4940736c0 0x7ff494075ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737a59c640 1 -- 172.21.15.64:0/1587220573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c15ed00 con 0x7f737c06cc30 2024-03-07T07:19:25.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7fccc4b99640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7fccac073820 0x7fccac075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f737ad9d640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f737c15bec0 0x7f737c152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:25.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7ff4b5da9640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff49406b520 0x7ff49406d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac000f60 tx=0x7ff4ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7fccbffff640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fccac06b5a0 0x7fccac06da60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccb8035000 tx=0x7fccb8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7f315337e640 1 -- 172.21.15.64:0/284882379 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f314c062c90 con 0x7f314c161e10 2024-03-07T07:19:25.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7fccc4b99640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7fccac073820 0x7fccac075c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.638+0000 7ff4b65aa640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7ff4940736c0 0x7ff494075ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== osd.5 v2:172.21.15.120:6800/349750424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fad8401ee50 con 0x7fad7c073870 2024-03-07T07:19:25.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7c8259c640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c054130 0x7f7c7c0cd870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c7c0 tx=0x7f7c7400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74018070 con 0x7f7c7c054130 2024-03-07T07:19:25.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c740074e0 con 0x7f7c7c054130 2024-03-07T07:19:25.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74004a20 con 0x7f7c7c054130 2024-03-07T07:19:25.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7c0d1090 con 0x7f7c7c054130 2024-03-07T07:19:25.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c0d1650 con 0x7f7c7c054130 2024-03-07T07:19:25.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f06a0038020 con 0x7f06a406bc20 2024-03-07T07:19:25.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.640+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== osd.0 v2:172.21.15.64:6800/382233877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff4a0033e50 con 0x7ff4940736c0 2024-03-07T07:19:25.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.640+0000 7f069a7fc640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f068806b5a0 0x7f068806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.640+0000 7f06a921e640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f068806b5a0 0x7f068806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.640+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c7c0057e0 con 0x7f7c7c054130 2024-03-07T07:19:25.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.640+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06a00acbc0 con 0x7f06a406bc20 2024-03-07T07:19:25.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.640+0000 7f06a921e640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f068806b5a0 0x7f068806da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f069c007fd0 tx=0x7f069c007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f737a59c640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c14dc50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f736c00e8a0 tx=0x7f736c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c017070 con 0x7f737c06cc30 2024-03-07T07:19:25.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c00c940 con 0x7f737c06cc30 2024-03-07T07:19:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.640+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== osd.6 v2:172.21.15.120:6824/2639829384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fccb000ce50 con 0x7fccac073820 2024-03-07T07:19:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00cd30 con 0x7f737c06cc30 2024-03-07T07:19:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7380816640 1 -- 172.21.15.64:0/1587220573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c15efe0 con 0x7f737c06cc30 2024-03-07T07:19:25.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.639+0000 7f7380816640 1 -- 172.21.15.64:0/1587220573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c153ab0 con 0x7f737c06cc30 2024-03-07T07:19:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f069a7fc640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f06880737c0 0x7f0688075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 --> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f06a0003820 con 0x7f06880737c0 2024-03-07T07:19:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a00acfb0 con 0x7f06a406bc20 2024-03-07T07:19:25.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f06aa220640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f06880737c0 0x7f0688075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7c7401b070 con 0x7f7c7c054130 2024-03-07T07:19:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f7c72ffd640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c6406b5a0 0x7f7c6406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c74089820 con 0x7f7c7c054130 2024-03-07T07:19:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f7c72ffd640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7f7c640737a0 0x7f7c64075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 --> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c64076270 con 0x7f7c640737a0 2024-03-07T07:19:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c740542d0 con 0x7f7c7c054130 2024-03-07T07:19:25.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f7c81d9b640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c6406b5a0 0x7f7c6406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f7c82d9d640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7f7c640737a0 0x7f7c64075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f7c81d9b640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c6406b5a0 0x7f7c6406da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c78034000 tx=0x7f7c78008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f7c82d9d640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7f7c640737a0 0x7f7c64075bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f31440050d0 con 0x7f314c161e10 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f3141ffb640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f313006b5a0 0x7f313006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.641+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f314408a780 con 0x7f314c161e10 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f3141ffb640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f31300737c0 0x7f3130075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 --> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3144009eb0 con 0x7f31300737c0 2024-03-07T07:19:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f314408ab70 con 0x7f314c161e10 2024-03-07T07:19:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f315111a640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f313006b5a0 0x7f313006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f3150919640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f31300737c0 0x7f3130075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f06aa220640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f06880737c0 0x7f0688075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.642+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f736c01a070 con 0x7f737c06cc30 2024-03-07T07:19:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f7380816640 1 -- 172.21.15.64:0/1587220573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f737c071db0 con 0x7f737c06cc30 2024-03-07T07:19:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f736affd640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f735c06b6d0 0x7f735c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f736c00caf0 con 0x7f737c06cc30 2024-03-07T07:19:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f736affd640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7f735c073870 0x7f735c075c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 --> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f735c076340 con 0x7f735c073870 2024-03-07T07:19:25.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.643+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f736c091050 con 0x7f737c06cc30 2024-03-07T07:19:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.644+0000 7f315111a640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f313006b5a0 0x7f313006da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3148009f20 tx=0x7f3148009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.644+0000 7f3150919640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f31300737c0 0x7f3130075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.644+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== osd.1 v2:172.21.15.64:6808/372208353 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7c64076270 con 0x7f7c640737a0 2024-03-07T07:19:25.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.644+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== osd.4 v2:172.21.15.120:6808/505090945 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f06a0003820 con 0x7f06880737c0 2024-03-07T07:19:25.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.644+0000 7f7379d9b640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f735c06b6d0 0x7f735c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.644+0000 7f737ad9d640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7f735c073870 0x7f735c075c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.644+0000 7f7379d9b640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f735c06b6d0 0x7f735c06db90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f737c14e550 tx=0x7f7370008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.645+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== osd.2 v2:172.21.15.64:6816/3037744413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3144009eb0 con 0x7f31300737c0 2024-03-07T07:19:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.645+0000 7f737ad9d640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7f735c073870 0x7f735c075c90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.650+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== osd.7 v2:172.21.15.120:6816/1578261849 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f735c076340 con 0x7f735c073870 2024-03-07T07:19:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.667+0000 7fccc65fc640 1 -- 172.21.15.64:0/1815940966 --> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fccc00b39e0 con 0x7fccac073820 2024-03-07T07:19:25.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.667+0000 7fccbcff9640 1 -- 172.21.15.64:0/1815940966 <== osd.6 v2:172.21.15.120:6824/2639829384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fccb400a800 con 0x7fccac073820 2024-03-07T07:19:25.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 -- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7fccac073820 msgr2=0x7fccac075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7fccac073820 0x7fccac075c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 -- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fccac06b5a0 msgr2=0x7fccac06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 --> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf740b1770 con 0x7fdf60073770 2024-03-07T07:19:25.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fccac06b5a0 0x7fccac06da60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccb8035000 tx=0x7fccb8030000 comp rx=0 tx=0).stop 2024-03-07T07:19:25.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 -- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccc0144860 msgr2=0x7fccc01f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 --2- 172.21.15.64:0/1815940966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccc0144860 0x7fccc01f87c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccb000b3f0 tx=0x7fccb000b8b0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fccc4b99640 1 -- 172.21.15.64:0/1815940966 reap_dead start 2024-03-07T07:19:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 -- 172.21.15.64:0/1815940966 shutdown_connections 2024-03-07T07:19:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.668+0000 7fcca67fc640 1 -- 172.21.15.64:0/1815940966 >> 172.21.15.64:0/1815940966 conn(0x7fccc006b040 msgr2=0x7fccc00ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.669+0000 7fcca67fc640 1 -- 172.21.15.64:0/1815940966 shutdown_connections 2024-03-07T07:19:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.669+0000 7fcca67fc640 1 -- 172.21.15.64:0/1815940966 wait complete. 2024-03-07T07:19:25.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.669+0000 7fdf72ffd640 1 -- 172.21.15.64:0/3917944578 <== osd.3 v2:172.21.15.64:6824/2270479097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf740b1770 con 0x7fdf60073770 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.670+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7fdf60073770 msgr2=0x7fdf60075b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.670+0000 7fdf70ff9640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7fdf60073770 0x7fdf60075b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.670+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fdf6006b5d0 msgr2=0x7fdf6006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.670+0000 7fdf70ff9640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fdf6006b5d0 0x7fdf6006da90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf68001760 tx=0x7fdf68000f50 comp rx=0 tx=0).stop 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.670+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf741599d0 msgr2=0x7fdf740ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.670+0000 7fdf70ff9640 1 --2- 172.21.15.64:0/3917944578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf741599d0 0x7fdf740ae3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf6402c230 tx=0x7fdf64004f00 comp rx=0 tx=0).stop 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.671+0000 7fdf7ad66640 1 -- 172.21.15.64:0/3917944578 reap_dead start 2024-03-07T07:19:25.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.671+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 --> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad900b37a0 con 0x7fad7c073870 2024-03-07T07:19:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.671+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 shutdown_connections 2024-03-07T07:19:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.671+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 >> 172.21.15.64:0/3917944578 conn(0x7fdf740b3120 msgr2=0x7fdf7406e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.671+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 shutdown_connections 2024-03-07T07:19:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.671+0000 7fdf70ff9640 1 -- 172.21.15.64:0/3917944578 wait complete. 2024-03-07T07:19:25.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.671+0000 7fad8cff9640 1 -- 172.21.15.64:0/3331023280 <== osd.5 v2:172.21.15.120:6800/349750424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad900b37a0 con 0x7fad7c073870 2024-03-07T07:19:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fad7c073870 msgr2=0x7fad7c075c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7fad9686b640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fad7c073870 0x7fad7c075c90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7ff4b6dab640 1 -- 172.21.15.64:0/2369561203 --> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff4a80057e0 con 0x7ff4940736c0 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.673+0000 7ff4a67fc640 1 -- 172.21.15.64:0/2369561203 <== osd.0 v2:172.21.15.64:6800/382233877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff4a80057e0 con 0x7ff4940736c0 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7c06b6d0 msgr2=0x7fad7c06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7fad9686b640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7c06b6d0 0x7fad7c06db90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad80009560 tx=0x7fad80009190 comp rx=0 tx=0).stop 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 msgr2=0x7fad9014f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7fad9686b640 1 --2- 172.21.15.64:0/3331023280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad90071b50 0x7fad9014f550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad8400d800 tx=0x7fad8400dcc0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.672+0000 7fad94e08640 1 -- 172.21.15.64:0/3331023280 reap_dead start 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.673+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 shutdown_connections 2024-03-07T07:19:25.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.673+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 >> 172.21.15.64:0/3331023280 conn(0x7fad900b3120 msgr2=0x7fad9009c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7ff493fff640 1 -- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7ff4940736c0 msgr2=0x7ff494075ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7ff493fff640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7ff4940736c0 0x7ff494075ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7ff493fff640 1 -- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff49406b520 msgr2=0x7ff49406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7ff493fff640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff49406b520 0x7ff49406d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac000f60 tx=0x7ff4ac009040 comp rx=0 tx=0).stop 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7ff493fff640 1 -- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80eac70 msgr2=0x7ff4a8031fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7ff493fff640 1 --2- 172.21.15.64:0/2369561203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80eac70 0x7ff4a8031fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0035040 tx=0x7ff4a00046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7ff4b65aa640 1 -- 172.21.15.64:0/2369561203 reap_dead start 2024-03-07T07:19:25.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 shutdown_connections 2024-03-07T07:19:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.674+0000 7fad9686b640 1 -- 172.21.15.64:0/3331023280 wait complete. 2024-03-07T07:19:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.675+0000 7ff493fff640 1 -- 172.21.15.64:0/2369561203 shutdown_connections 2024-03-07T07:19:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.675+0000 7ff493fff640 1 -- 172.21.15.64:0/2369561203 >> 172.21.15.64:0/2369561203 conn(0x7ff4a801e6c0 msgr2=0x7ff4a80e77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.675+0000 7ff493fff640 1 -- 172.21.15.64:0/2369561203 shutdown_connections 2024-03-07T07:19:25.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.675+0000 7ff493fff640 1 -- 172.21.15.64:0/2369561203 wait complete. 2024-03-07T07:19:25.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.676+0000 7f06abc83640 1 -- 172.21.15.64:0/1881716492 --> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f06a4153560 con 0x7f06880737c0 2024-03-07T07:19:25.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.676+0000 7f069a7fc640 1 -- 172.21.15.64:0/1881716492 <== osd.4 v2:172.21.15.120:6808/505090945 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f06940092e0 con 0x7f06880737c0 2024-03-07T07:19:25.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 --> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c7c0397e0 con 0x7f7c640737a0 2024-03-07T07:19:25.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f315337e640 1 -- 172.21.15.64:0/284882379 --> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f314c0d3e00 con 0x7f31300737c0 2024-03-07T07:19:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f0687fff640 1 -- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f06880737c0 msgr2=0x7f0688075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f0687fff640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f06880737c0 0x7f0688075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f0687fff640 1 -- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f068806b5a0 msgr2=0x7f068806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f7c72ffd640 1 -- 172.21.15.64:0/3096124966 <== osd.1 v2:172.21.15.64:6808/372208353 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7c7c0397e0 con 0x7f7c640737a0 2024-03-07T07:19:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f0687fff640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f068806b5a0 0x7f068806da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f069c007fd0 tx=0x7f069c007d40 comp rx=0 tx=0).stop 2024-03-07T07:19:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f0687fff640 1 -- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 msgr2=0x7f06a41637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f0687fff640 1 --2- 172.21.15.64:0/1881716492 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f06a406bc20 0x7f06a41637a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a00046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.677+0000 7f3141ffb640 1 -- 172.21.15.64:0/284882379 <== osd.2 v2:172.21.15.64:6816/3037744413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f314c0d3e00 con 0x7f31300737c0 2024-03-07T07:19:25.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f06aa220640 1 -- 172.21.15.64:0/1881716492 reap_dead start 2024-03-07T07:19:25.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7f7c640737a0 msgr2=0x7f7c64075bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f7c8359e640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7f7c640737a0 0x7f7c64075bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f0687fff640 1 -- 172.21.15.64:0/1881716492 shutdown_connections 2024-03-07T07:19:25.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f0687fff640 1 -- 172.21.15.64:0/1881716492 >> 172.21.15.64:0/1881716492 conn(0x7f06a406d4c0 msgr2=0x7f06a4072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f0687fff640 1 -- 172.21.15.64:0/1881716492 shutdown_connections 2024-03-07T07:19:25.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f0687fff640 1 -- 172.21.15.64:0/1881716492 wait complete. 2024-03-07T07:19:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c6406b5a0 msgr2=0x7f7c6406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f7c8359e640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c6406b5a0 0x7f7c6406da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c78034000 tx=0x7f7c78008030 comp rx=0 tx=0).stop 2024-03-07T07:19:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c054130 msgr2=0x7f7c7c0cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f7c8359e640 1 --2- 172.21.15.64:0/3096124966 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c7c054130 0x7f7c7c0cd870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c7c0 tx=0x7f7c7400cbc0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f7c82d9d640 1 -- 172.21.15.64:0/3096124966 reap_dead start 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f312f7fe640 1 -- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f31300737c0 msgr2=0x7f3130075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.678+0000 7f312f7fe640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f31300737c0 0x7f3130075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 -- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f313006b5a0 msgr2=0x7f313006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f313006b5a0 0x7f313006da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3148009f20 tx=0x7f3148009240 comp rx=0 tx=0).stop 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 -- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c161e10 msgr2=0x7f314c156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 --2- 172.21.15.64:0/284882379 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f314c161e10 0x7f314c156580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f314400c730 tx=0x7f314400cb30 comp rx=0 tx=0).stop 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f315191b640 1 -- 172.21.15.64:0/284882379 reap_dead start 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 -- 172.21.15.64:0/284882379 shutdown_connections 2024-03-07T07:19:25.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 -- 172.21.15.64:0/284882379 >> 172.21.15.64:0/284882379 conn(0x7f314c0d2270 msgr2=0x7f314c0d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 shutdown_connections 2024-03-07T07:19:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 >> 172.21.15.64:0/3096124966 conn(0x7f7c7c0559e0 msgr2=0x7f7c7c039ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 shutdown_connections 2024-03-07T07:19:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f7c8359e640 1 -- 172.21.15.64:0/3096124966 wait complete. 2024-03-07T07:19:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 -- 172.21.15.64:0/284882379 shutdown_connections 2024-03-07T07:19:25.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.679+0000 7f312f7fe640 1 -- 172.21.15.64:0/284882379 wait complete. 2024-03-07T07:19:25.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.680+0000 7f7380816640 1 -- 172.21.15.64:0/1587220573 --> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f737c15f1b0 con 0x7f735c073870 2024-03-07T07:19:25.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.681+0000 7f736affd640 1 -- 172.21.15.64:0/1587220573 <== osd.7 v2:172.21.15.120:6816/1578261849 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f737c15f1b0 con 0x7f735c073870 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.682+0000 7f7368ff9640 1 -- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7f735c073870 msgr2=0x7f735c075c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.682+0000 7f7368ff9640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7f735c073870 0x7f735c075c90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.682+0000 7f7368ff9640 1 -- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f735c06b6d0 msgr2=0x7f735c06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.682+0000 7f7368ff9640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f735c06b6d0 0x7f735c06db90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f737c14e550 tx=0x7f7370008040 comp rx=0 tx=0).stop 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.682+0000 7f7368ff9640 1 -- 172.21.15.64:0/1587220573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 msgr2=0x7f737c14dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.682+0000 7f7368ff9640 1 --2- 172.21.15.64:0/1587220573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737c06cc30 0x7f737c14dc50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f736c00e8a0 tx=0x7f736c00ed60 comp rx=0 tx=0).stop 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.682+0000 7f737ad9d640 1 -- 172.21.15.64:0/1587220573 reap_dead start 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.683+0000 7f7368ff9640 1 -- 172.21.15.64:0/1587220573 shutdown_connections 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.683+0000 7f7368ff9640 1 -- 172.21.15.64:0/1587220573 >> 172.21.15.64:0/1587220573 conn(0x7f737c070400 msgr2=0x7f737c15cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.684+0000 7f7368ff9640 1 -- 172.21.15.64:0/1587220573 shutdown_connections 2024-03-07T07:19:25.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.684+0000 7f7368ff9640 1 -- 172.21.15.64:0/1587220573 wait complete. 2024-03-07T07:19:25.736 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:25.736 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-07T07:19:25.737 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-03-07T07:19:25.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-07T07:19:25.738 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2024-03-07T07:19:25.739 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:25.739 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-07T07:19:25.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-07T07:19:25.740 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:25.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-07T07:19:25.741 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:25.742 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-07T07:19:25.742 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-03-07T07:19:25.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-07T07:19:25.743 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:25.744 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-07T07:19:25.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.947+0000 7f30c3e34640 1 Processor -- start 2024-03-07T07:19:25.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.947+0000 7f70e6e34640 1 Processor -- start 2024-03-07T07:19:25.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.947+0000 7f70e6e34640 1 -- start start 2024-03-07T07:19:25.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.947+0000 7f30c3e34640 1 -- start start 2024-03-07T07:19:25.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c3e34640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f30bc0cf6d0 0x7f30bc06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c3e34640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc162960 0x7f30bc071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c3e34640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f30bc071e60 0x7f30bc0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c3e34640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30bc163170 con 0x7f30bc071e60 2024-03-07T07:19:25.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c3e34640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30bc163450 con 0x7f30bc0cf6d0 2024-03-07T07:19:25.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c3e34640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc163730 con 0x7f30bc162960 2024-03-07T07:19:25.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c23d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f30bc071e60 0x7f30bc0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c23d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f30bc071e60 0x7f30bc0cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34406/0 (socket says 172.21.15.64:34406) 2024-03-07T07:19:25.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c23d1640 1 -- 172.21.15.64:0/2409925255 learned_addr learned my addr 172.21.15.64:0/2409925255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c1bd0640 1 --2- 172.21.15.64:0/2409925255 >> v2:172.21.15.120:3300/0 conn(0x7f30bc0cf6d0 0x7f30bc06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c13cf640 1 --2- 172.21.15.64:0/2409925255 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc162960 0x7f30bc071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f70e6e34640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f70d80df220 0x7f70d8038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c23d1640 1 -- 172.21.15.64:0/2409925255 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc162960 msgr2=0x7f30bc071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70e6e34640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f70d8053c70 0x7f70d804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c23d1640 1 --2- 172.21.15.64:0/2409925255 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc162960 0x7f30bc071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c23d1640 1 -- 172.21.15.64:0/2409925255 >> v2:172.21.15.120:3300/0 conn(0x7f30bc0cf6d0 msgr2=0x7f30bc06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.949+0000 7f30c23d1640 1 --2- 172.21.15.64:0/2409925255 >> v2:172.21.15.120:3300/0 conn(0x7f30bc0cf6d0 0x7f30bc06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70e6e34640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d804ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70dffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f70d8053c70 0x7f70d804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70dffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f70d8053c70 0x7f70d804fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34258/0 (socket says 172.21.15.64:34258) 2024-03-07T07:19:25.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f30c23d1640 1 -- 172.21.15.64:0/2409925255 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc0b2ec0 con 0x7f30bc071e60 2024-03-07T07:19:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70dffff640 1 -- 172.21.15.64:0/2806693429 learned_addr learned my addr 172.21.15.64:0/2806693429 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d80544c0 con 0x7f70d80df220 2024-03-07T07:19:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70e53d1640 1 --2- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d804ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d80541e0 con 0x7f70d8053c70 2024-03-07T07:19:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8005f30 con 0x7f70d8033ef0 2024-03-07T07:19:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70e4bd0640 1 --2- 172.21.15.64:0/2806693429 >> v2:172.21.15.64:3300/0 conn(0x7f70d80df220 0x7f70d8038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70dffff640 1 -- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 msgr2=0x7f70d804ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70dffff640 1 --2- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d804ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.950+0000 7f70dffff640 1 -- 172.21.15.64:0/2806693429 >> v2:172.21.15.64:3300/0 conn(0x7f70d80df220 msgr2=0x7f70d8038080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7ff7c18d0640 1 Processor -- start 2024-03-07T07:19:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f6bb4a25640 1 Processor -- start 2024-03-07T07:19:25.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7ff7c18d0640 1 -- start start 2024-03-07T07:19:25.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f51438dd640 1 Processor -- start 2024-03-07T07:19:25.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f70dffff640 1 --2- 172.21.15.64:0/2806693429 >> v2:172.21.15.64:3300/0 conn(0x7f70d80df220 0x7f70d8038080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f70dffff640 1 -- 172.21.15.64:0/2806693429 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8006210 con 0x7f70d8053c70 2024-03-07T07:19:25.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f70e53d1640 1 --2- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d804ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:25.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f70e4bd0640 1 --2- 172.21.15.64:0/2806693429 >> v2:172.21.15.64:3300/0 conn(0x7f70d80df220 0x7f70d8038080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:25.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f6bb4a25640 1 -- start start 2024-03-07T07:19:25.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f51438dd640 1 -- start start 2024-03-07T07:19:25.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f6bb4a25640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dfef0 0x7f6ba80085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f2649a12640 1 Processor -- start 2024-03-07T07:19:25.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.951+0000 7f2649a12640 1 -- start start 2024-03-07T07:19:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f6bb4a25640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba8056590 con 0x7f6ba80e02d0 2024-03-07T07:19:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f6bb4a25640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba8056870 con 0x7f6ba80e42d0 2024-03-07T07:19:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f6bb4a25640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80094b0 con 0x7f6ba80dfef0 2024-03-07T07:19:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f8d3fb78640 1 Processor -- start 2024-03-07T07:19:25.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f8d3fb78640 1 -- start start 2024-03-07T07:19:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7c18d0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b403e870 0x7ff7b4053c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7c18d0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff7b4054160 0x7ff7b4006900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7c18d0640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b40585a0 con 0x7ff7b40e30f0 2024-03-07T07:19:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7c18d0640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b4058880 con 0x7ff7b4054160 2024-03-07T07:19:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f2649a12640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26441447a0 0x7f26440ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7c18d0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4058b60 con 0x7ff7b403e870 2024-03-07T07:19:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f6baffff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f6ba80e42d0 0x7f6ba8006ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:47800/0 (socket says 172.21.15.64:47800) 2024-03-07T07:19:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f6baffff640 1 -- 172.21.15.64:0/4032104534 learned_addr learned my addr 172.21.15.64:0/4032104534 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f30c23d1640 1 --2- 172.21.15.64:0/2409925255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc071e60 0x7f30bc0cb670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30bc06b1d0 tx=0x7f30b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e839a46a9fe5512 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7baffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff7b40e30f0 0x7ff7b4057dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48696/0 (socket says 172.21.15.64:48696) 2024-03-07T07:19:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7baffd640 1 -- 172.21.15.64:0/4020252056 learned_addr learned my addr 172.21.15.64:0/4020252056 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:25.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7ff7ba7fc640 1 --2- 172.21.15.64:0/4020252056 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b403e870 0x7ff7b4053c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2649a12640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f264415c480 0x7f264415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f92e0b37640 1 Processor -- start 2024-03-07T07:19:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.952+0000 7f6baf7fe640 1 --2- 172.21.15.64:0/4032104534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dfef0 0x7f6ba80085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f6bae7fc640 1 -- 172.21.15.64:0/4032104534 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404792420 0 0) 0x7f6ba8056870 con 0x7f6ba80e42d0 2024-03-07T07:19:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f6bae7fc640 1 -- 172.21.15.64:0/4032104534 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba0007930 con 0x7f6ba80e42d0 2024-03-07T07:19:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f70dffff640 1 --2- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d8053c70 0x7f70d804fa90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b3f0 tx=0x7f70cc02dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=521c0d4396e6bad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2648a10640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26441447a0 0x7f26440ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2648a10640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f26441447a0 0x7f26440ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34434/0 (socket says 172.21.15.64:34434) 2024-03-07T07:19:25.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2648a10640 1 -- 172.21.15.64:0/2875321371 learned_addr learned my addr 172.21.15.64:0/2875321371 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f30c0bce640 1 -- 172.21.15.64:0/2409925255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b402ca80 con 0x7f30bc071e60 2024-03-07T07:19:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f6baf7fe640 1 -- 172.21.15.64:0/4032104534 >> v1:172.21.15.120:6789/0 conn(0x7f6ba80e42d0 legacy=0x7f6ba8006ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264415c8e0 con 0x7f26440b2830 2024-03-07T07:19:25.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264406e070 con 0x7f26441447a0 2024-03-07T07:19:25.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f30c0bce640 1 -- 172.21.15.64:0/2409925255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b402cc20 con 0x7f30bc071e60 2024-03-07T07:19:25.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3fb78640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8d300e2db0 0x7f8d300e3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3fb78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d30006fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3d113640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8d300e2db0 0x7f8d300e3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3d113640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8d300e2db0 0x7f8d300e3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34262/0 (socket says 172.21.15.64:34262) 2024-03-07T07:19:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2643fff640 1 --2- 172.21.15.64:0/2875321371 >> v2:172.21.15.120:3300/0 conn(0x7f264415c480 0x7f264415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264415bae0 con 0x7f264415c480 2024-03-07T07:19:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f51438dd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f513c0cf140 0x7f513c0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3d113640 1 -- 172.21.15.64:0/2167780485 learned_addr learned my addr 172.21.15.64:0/2167780485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d30054670 con 0x7f8d300df250 2024-03-07T07:19:25.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d300e3750 con 0x7f8d300e2db0 2024-03-07T07:19:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30054420 con 0x7f8d30006b10 2024-03-07T07:19:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f51438dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f513c06de60 0x7f513c163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f8d3e115640 1 --2- 172.21.15.64:0/2167780485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d30006fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f5141679640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f513c0cf140 0x7f513c0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f51438dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c06c0d0 con 0x7f513c0722e0 2024-03-07T07:19:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f51438dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513c06c370 con 0x7f513c06de60 2024-03-07T07:19:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f51438dd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513c06c650 con 0x7f513c0cf140 2024-03-07T07:19:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f5141679640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f513c0cf140 0x7f513c0b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34268/0 (socket says 172.21.15.64:34268) 2024-03-07T07:19:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f5141679640 1 -- 172.21.15.64:0/2140948428 learned_addr learned my addr 172.21.15.64:0/2140948428 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f70df7fe640 1 -- 172.21.15.64:0/2806693429 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc03b070 con 0x7f70d8053c70 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f30c3e34640 1 -- 172.21.15.64:0/2409925255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc071e60 msgr2=0x7f30bc0cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f30c3e34640 1 --2- 172.21.15.64:0/2409925255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc071e60 0x7f30bc0cb670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30bc06b1d0 tx=0x7f30b402bee0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7ff7bb7fe640 1 --2- 172.21.15.64:0/4020252056 >> v2:172.21.15.64:3300/0 conn(0x7ff7b4054160 0x7ff7b4006900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7ff7b9ffb640 1 -- 172.21.15.64:0/4020252056 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268924830 0 0) 0x7ff7b40585a0 con 0x7ff7b40e30f0 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7ff7b9ffb640 1 -- 172.21.15.64:0/4020252056 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a4003520 con 0x7ff7b40e30f0 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7ff7bb7fe640 1 -- 172.21.15.64:0/4020252056 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b403e870 msgr2=0x7ff7b4053c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7ff7bb7fe640 1 --2- 172.21.15.64:0/4020252056 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b403e870 0x7ff7b4053c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7ff7bb7fe640 1 -- 172.21.15.64:0/4020252056 >> v1:172.21.15.64:6789/0 conn(0x7ff7b40e30f0 legacy=0x7ff7b4057dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7ff7bb7fe640 1 -- 172.21.15.64:0/4020252056 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40063e0 con 0x7ff7b4054160 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f92e0b37640 1 -- start start 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92e0b37640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f92d4034020 0x7f92d404fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92e0b37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4006a50 0x7f92d4006ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92db7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f92d404b3c0 0x7f92d404f4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:47834/0 (socket says 172.21.15.64:47834) 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92e0b37640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d4054c10 con 0x7f92d404b3c0 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92e0b37640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d40062e0 con 0x7f92d4034020 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92e0b37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4050120 con 0x7f92d4006a50 2024-03-07T07:19:25.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f6baf7fe640 1 -- 172.21.15.64:0/4032104534 >> v1:172.21.15.64:6789/0 conn(0x7f6ba80e02d0 legacy=0x7f6ba80e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f6baf7fe640 1 -- 172.21.15.64:0/4032104534 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8008e60 con 0x7f6ba80dfef0 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537955497 0 0) 0x7f264415c8e0 con 0x7f26440b2830 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.953+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f262c003520 con 0x7f26440b2830 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 <== mon.1 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1109558412 0 0) 0x7f262c003520 con 0x7f26440b2830 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 >> v2:172.21.15.120:3300/0 conn(0x7f264415c480 msgr2=0x7f264415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 --2- 172.21.15.64:0/2875321371 >> v2:172.21.15.120:3300/0 conn(0x7f264415c480 0x7f264415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 >> v2:172.21.15.64:3300/0 conn(0x7f26441447a0 msgr2=0x7f26440ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 --2- 172.21.15.64:0/2875321371 >> v2:172.21.15.64:3300/0 conn(0x7f26441447a0 0x7f26440ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264415bdc0 con 0x7f26440b2830 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 <== mon.1 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f2634001650 con 0x7f26440b2830 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 >> v1:172.21.15.120:6789/0 conn(0x7f26440b2830 legacy=0x7f264406e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f262c0039f0 0x7f262c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c004800 0x7f262c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c005230 0x7f262c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264406e070 con 0x7f262c004800 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264415bae0 con 0x7f262c005230 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c00e420 con 0x7f262c0039f0 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f2648a10640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f262c0039f0 0x7f262c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f70df7fe640 1 -- 172.21.15.64:0/2806693429 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc02ea70 con 0x7f70d8053c70 2024-03-07T07:19:25.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f70df7fe640 1 -- 172.21.15.64:0/2806693429 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc02ee40 con 0x7f70d8053c70 2024-03-07T07:19:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92dbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4006a50 0x7f92d4006ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92daffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f92d4034020 0x7f92d404fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92dbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4006a50 0x7f92d4006ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:56986/0 (socket says 172.21.15.64:56986) 2024-03-07T07:19:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f92dbfff640 1 -- 172.21.15.64:0/1265391073 learned_addr learned my addr 172.21.15.64:0/1265391073 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f2649211640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c005230 0x7f262c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f5133fff640 1 -- 172.21.15.64:0/2140948428 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787540956 0 0) 0x7f513c06c0d0 con 0x7f513c0722e0 2024-03-07T07:19:25.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f5133fff640 1 -- 172.21.15.64:0/2140948428 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128005730 con 0x7f513c0722e0 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f5141679640 1 -- 172.21.15.64:0/2140948428 >> v2:172.21.15.64:3300/0 conn(0x7f513c06de60 msgr2=0x7f513c163d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f2648a10640 1 -- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c005230 msgr2=0x7f262c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f2648a10640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c005230 0x7f262c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f2648a10640 1 -- 172.21.15.64:0/2875321371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c004800 msgr2=0x7f262c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f2648a10640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f262c004800 0x7f262c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f30c3e34640 1 -- 172.21.15.64:0/2409925255 shutdown_connections 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f30c3e34640 1 --2- 172.21.15.64:0/2409925255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc071e60 0x7f30bc0cb670 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f30c3e34640 1 --2- 172.21.15.64:0/2409925255 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc162960 0x7f30bc071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f30c3e34640 1 --2- 172.21.15.64:0/2409925255 >> v2:172.21.15.120:3300/0 conn(0x7f30bc0cf6d0 0x7f30bc06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f30c3e34640 1 -- 172.21.15.64:0/2409925255 >> 172.21.15.64:0/2409925255 conn(0x7f30bc0cfb90 msgr2=0x7f30bc0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f2648a10640 1 -- 172.21.15.64:0/2875321371 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264415bdc0 con 0x7f262c0039f0 2024-03-07T07:19:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f8d3e115640 1 -- 172.21.15.64:0/2167780485 >> v2:172.21.15.120:3300/0 conn(0x7f8d300e2db0 msgr2=0x7f8d300e3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f8d3e115640 1 --2- 172.21.15.64:0/2167780485 >> v2:172.21.15.120:3300/0 conn(0x7f8d300e2db0 0x7f8d300e3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f8d3e115640 1 -- 172.21.15.64:0/2167780485 >> v1:172.21.15.64:6789/0 conn(0x7f8d300df250 legacy=0x7f8d3003a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.954+0000 7f8d3e115640 1 -- 172.21.15.64:0/2167780485 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30054950 con 0x7f8d30006b10 2024-03-07T07:19:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f92daffd640 1 -- 172.21.15.64:0/1265391073 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4006a50 msgr2=0x7f92d4006ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f92daffd640 1 --2- 172.21.15.64:0/1265391073 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4006a50 0x7f92d4006ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f92daffd640 1 -- 172.21.15.64:0/1265391073 >> v1:172.21.15.120:6789/0 conn(0x7f92d404b3c0 legacy=0x7f92d404f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f92daffd640 1 -- 172.21.15.64:0/1265391073 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4053a50 con 0x7f92d4034020 2024-03-07T07:19:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f92db7fe640 1 --1- 172.21.15.64:0/1265391073 >> v1:172.21.15.120:6789/0 conn(0x7f92d404b3c0 0x7f92d404f4d0 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-07T07:19:25.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f6baf7fe640 1 --2- 172.21.15.64:0/4032104534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dfef0 0x7f6ba80085d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009750 tx=0x7f6b9c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6290c8e4967cef server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f5133fff640 1 -- 172.21.15.64:0/2140948428 <== mon.1 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750569708 0 0) 0x7f5128005730 con 0x7f513c0722e0 2024-03-07T07:19:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7ff7bb7fe640 1 --2- 172.21.15.64:0/4020252056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4006900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc050ba0 tx=0x7ff7bc0a6c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e99c9902acf80f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f30c3e34640 1 -- 172.21.15.64:0/2409925255 shutdown_connections 2024-03-07T07:19:25.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f6bae7fc640 1 -- 172.21.15.64:0/4032104534 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c039070 con 0x7f6ba80dfef0 2024-03-07T07:19:25.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d8053c70 msgr2=0x7f70d804fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d8053c70 0x7f70d804fa90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b3f0 tx=0x7f70cc02dcc0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f6bae7fc640 1 -- 172.21.15.64:0/4032104534 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0314f0 con 0x7f6ba80dfef0 2024-03-07T07:19:25.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f6bae7fc640 1 -- 172.21.15.64:0/4032104534 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4001670 con 0x7f6ba80dfef0 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 shutdown_connections 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d804ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2806693429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d8053c70 0x7f70d804fa90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2806693429 >> v2:172.21.15.64:3300/0 conn(0x7f70d80df220 0x7f70d8038080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 >> 172.21.15.64:0/2806693429 conn(0x7f70d8009fb0 msgr2=0x7f70d800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 shutdown_connections 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f70e6e34640 1 -- 172.21.15.64:0/2806693429 wait complete. 2024-03-07T07:19:25.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f2649211640 1 -- 172.21.15.64:0/2875321371 reap_dead start 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7ff7b9ffb640 1 -- 172.21.15.64:0/4020252056 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0b4070 con 0x7ff7b4054160 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7ff7b9ffb640 1 -- 172.21.15.64:0/4020252056 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc0a7870 con 0x7ff7b4054160 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7ff7b9ffb640 1 -- 172.21.15.64:0/4020252056 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0b4070 con 0x7ff7b4054160 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f5141e7a640 1 --2- 172.21.15.64:0/2140948428 >> v2:172.21.15.64:3300/0 conn(0x7f513c06de60 0x7f513c163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f5141679640 1 --2- 172.21.15.64:0/2140948428 >> v2:172.21.15.64:3300/0 conn(0x7f513c06de60 0x7f513c163d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f5141679640 1 -- 172.21.15.64:0/2140948428 >> v1:172.21.15.120:6789/0 conn(0x7f513c0722e0 legacy=0x7f513c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f5141679640 1 -- 172.21.15.64:0/2140948428 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c06c930 con 0x7f513c0cf140 2024-03-07T07:19:25.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f5141e7a640 1 --2- 172.21.15.64:0/2140948428 >> v2:172.21.15.64:3300/0 conn(0x7f513c06de60 0x7f513c163d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f8d3e115640 1 --2- 172.21.15.64:0/2167780485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d30006fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8d280097a0 tx=0x7f8d2802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599d4dcce649c6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f2648a10640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f262c0039f0 0x7f262c003e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26380098d0 tx=0x7f262c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e57722624c731a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f92daffd640 1 --2- 172.21.15.64:0/1265391073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4034020 0x7f92d404fbe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f92c80097a0 tx=0x7f92c802b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c591eadac92ef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263800f040 con 0x7f262c0039f0 2024-03-07T07:19:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263802f630 con 0x7f262c0039f0 2024-03-07T07:19:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f26437fe640 1 -- 172.21.15.64:0/2875321371 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263802fa00 con 0x7f262c0039f0 2024-03-07T07:19:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.955+0000 7f30c3e34640 1 -- 172.21.15.64:0/2409925255 wait complete. 2024-03-07T07:19:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f6bb4a25640 1 -- 172.21.15.64:0/4032104534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dfef0 msgr2=0x7f6ba80085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f6bb4a25640 1 --2- 172.21.15.64:0/4032104534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dfef0 0x7f6ba80085d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009750 tx=0x7f6b9c02b790 comp rx=0 tx=0).stop 2024-03-07T07:19:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f92da7fc640 1 -- 172.21.15.64:0/1265391073 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8037070 con 0x7f92d4034020 2024-03-07T07:19:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f92da7fc640 1 -- 172.21.15.64:0/1265391073 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c8004230 con 0x7f92d4034020 2024-03-07T07:19:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f92da7fc640 1 -- 172.21.15.64:0/1265391073 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8003610 con 0x7f92d4034020 2024-03-07T07:19:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f8d3c912640 1 -- 172.21.15.64:0/2167780485 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28031780 con 0x7f8d30006b10 2024-03-07T07:19:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f8d3c912640 1 -- 172.21.15.64:0/2167780485 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d28037070 con 0x7f8d30006b10 2024-03-07T07:19:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f5141679640 1 --2- 172.21.15.64:0/2140948428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c0cf140 0x7f513c0b0ca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f513c0cddc0 tx=0x7f513802ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12479e9b6eddbfda server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 -- 172.21.15.64:0/4020252056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 msgr2=0x7ff7b4006900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/4020252056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4006900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc050ba0 tx=0x7ff7bc0a6c10 comp rx=0 tx=0).stop 2024-03-07T07:19:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f6bb4a25640 1 -- 172.21.15.64:0/4032104534 shutdown_connections 2024-03-07T07:19:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f6bb4a25640 1 --2- 172.21.15.64:0/4032104534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dfef0 0x7f6ba80085d0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009750 tx=0x7f6b9c02b790 comp rx=0 tx=0).stop 2024-03-07T07:19:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f6bb4a25640 1 -- 172.21.15.64:0/4032104534 >> 172.21.15.64:0/4032104534 conn(0x7f6ba801ef40 msgr2=0x7f6ba8055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 -- 172.21.15.64:0/4020252056 shutdown_connections 2024-03-07T07:19:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/4020252056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4006900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/4020252056 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b403e870 0x7ff7b4053c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 -- 172.21.15.64:0/4020252056 >> 172.21.15.64:0/4020252056 conn(0x7ff7b4055a10 msgr2=0x7ff7b4055e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f5133fff640 1 -- 172.21.15.64:0/2140948428 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138034070 con 0x7f513c0cf140 2024-03-07T07:19:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f5133fff640 1 -- 172.21.15.64:0/2140948428 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5138030650 con 0x7f513c0cf140 2024-03-07T07:19:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f5133fff640 1 -- 172.21.15.64:0/2140948428 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138034070 con 0x7f513c0cf140 2024-03-07T07:19:25.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f6bb4a25640 1 -- 172.21.15.64:0/4032104534 shutdown_connections 2024-03-07T07:19:25.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.956+0000 7f8d3c912640 1 -- 172.21.15.64:0/2167780485 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28031b80 con 0x7f8d30006b10 2024-03-07T07:19:25.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f6bb4a25640 1 -- 172.21.15.64:0/4032104534 wait complete. 2024-03-07T07:19:25.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f70e6e34640 1 Processor -- start 2024-03-07T07:19:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f262c0039f0 msgr2=0x7f262c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 msgr2=0x7f8d30006fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/2167780485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d30006fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8d280097a0 tx=0x7f8d2802b820 comp rx=0 tx=0).stop 2024-03-07T07:19:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f70e6e34640 1 -- start start 2024-03-07T07:19:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 -- 172.21.15.64:0/4020252056 shutdown_connections 2024-03-07T07:19:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f2649a12640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f262c0039f0 0x7f262c003e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26380098d0 tx=0x7f262c005830 comp rx=0 tx=0).stop 2024-03-07T07:19:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 shutdown_connections 2024-03-07T07:19:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 shutdown_connections 2024-03-07T07:19:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/2167780485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d30006fa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/2167780485 >> v2:172.21.15.120:3300/0 conn(0x7f8d300e2db0 0x7f8d300e3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 >> 172.21.15.64:0/2167780485 conn(0x7f8d3000a430 msgr2=0x7f8d3000a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f92e0b37640 1 -- 172.21.15.64:0/1265391073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4034020 msgr2=0x7f92d404fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f92e0b37640 1 --2- 172.21.15.64:0/1265391073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4034020 0x7f92d404fbe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f92c80097a0 tx=0x7f92c802b760 comp rx=0 tx=0).stop 2024-03-07T07:19:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f2649a12640 1 --2- 172.21.15.64:0/2875321371 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f262c0039f0 0x7f262c003e30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 >> 172.21.15.64:0/2875321371 conn(0x7f2644067090 msgr2=0x7f264406b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 shutdown_connections 2024-03-07T07:19:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7f2649a12640 1 -- 172.21.15.64:0/2875321371 wait complete. 2024-03-07T07:19:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f30c3e34640 1 Processor -- start 2024-03-07T07:19:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f92e0b37640 1 -- 172.21.15.64:0/1265391073 shutdown_connections 2024-03-07T07:19:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f92e0b37640 1 --2- 172.21.15.64:0/1265391073 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4006a50 0x7f92d4006ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f92e0b37640 1 --2- 172.21.15.64:0/1265391073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4034020 0x7f92d404fbe0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f92e0b37640 1 -- 172.21.15.64:0/1265391073 >> 172.21.15.64:0/1265391073 conn(0x7f92d400a400 msgr2=0x7f92d400a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.957+0000 7ff7c18d0640 1 -- 172.21.15.64:0/4020252056 wait complete. 2024-03-07T07:19:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f51438dd640 1 -- 172.21.15.64:0/2140948428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c0cf140 msgr2=0x7f513c0b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f51438dd640 1 --2- 172.21.15.64:0/2140948428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c0cf140 0x7f513c0b0ca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f513c0cddc0 tx=0x7f513802ba10 comp rx=0 tx=0).stop 2024-03-07T07:19:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f30c3e34640 1 -- start start 2024-03-07T07:19:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 shutdown_connections 2024-03-07T07:19:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f51438dd640 1 -- 172.21.15.64:0/2140948428 shutdown_connections 2024-03-07T07:19:25.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f51438dd640 1 --2- 172.21.15.64:0/2140948428 >> v2:172.21.15.64:3300/0 conn(0x7f513c06de60 0x7f513c163d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f92e0b37640 1 -- 172.21.15.64:0/1265391073 shutdown_connections 2024-03-07T07:19:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f51438dd640 1 --2- 172.21.15.64:0/2140948428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c0cf140 0x7f513c0b0ca0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f51438dd640 1 -- 172.21.15.64:0/2140948428 >> 172.21.15.64:0/2140948428 conn(0x7f513c0b1d00 msgr2=0x7f513c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f51438dd640 1 -- 172.21.15.64:0/2140948428 shutdown_connections 2024-03-07T07:19:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f92e0b37640 1 -- 172.21.15.64:0/1265391073 wait complete. 2024-03-07T07:19:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e6e34640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d80e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e6e34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d8053c70 0x7f70d80e2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e6e34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d80df220 0x7f70d8175030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e6e34640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8037a90 con 0x7f70d8053c70 2024-03-07T07:19:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e6e34640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8037910 con 0x7f70d80df220 2024-03-07T07:19:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e6e34640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d80df600 con 0x7f70d8033ef0 2024-03-07T07:19:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.958+0000 7f8d3fb78640 1 -- 172.21.15.64:0/2167780485 wait complete. 2024-03-07T07:19:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e53d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d80df220 0x7f70d8175030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7ff7c18d0640 1 Processor -- start 2024-03-07T07:19:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f2649a12640 1 Processor -- start 2024-03-07T07:19:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e53d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d80df220 0x7f70d8175030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34298/0 (socket says 172.21.15.64:34298) 2024-03-07T07:19:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f70e53d1640 1 -- 172.21.15.64:0/2932783724 learned_addr learned my addr 172.21.15.64:0/2932783724 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f30c3e34640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc0cf6d0 0x7f30bc1f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f51438dd640 1 -- 172.21.15.64:0/2140948428 wait complete. 2024-03-07T07:19:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c3e34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc162960 0x7f30bc141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c1bd0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc0cf6d0 0x7f30bc1f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f70e53d1640 1 -- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 msgr2=0x7f70d80e1ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-07T07:19:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f70e53d1640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d80e1ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f70e53d1640 1 -- 172.21.15.64:0/2932783724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d8053c70 msgr2=0x7f70d80e2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f70e53d1640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d8053c70 0x7f70d80e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f70e53d1640 1 -- 172.21.15.64:0/2932783724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d804e7a0 con 0x7f70d80df220 2024-03-07T07:19:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c1bd0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc0cf6d0 0x7f30bc1f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:57014/0 (socket says 172.21.15.64:57014) 2024-03-07T07:19:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f70e53d1640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d80df220 0x7f70d8175030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f70d400d5d0 tx=0x7f70d400da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c13cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc162960 0x7f30bc141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c13cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc162960 0x7f30bc141270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34506/0 (socket says 172.21.15.64:34506) 2024-03-07T07:19:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f8d3fb78640 1 Processor -- start 2024-03-07T07:19:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c1bd0640 1 -- 172.21.15.64:0/2559259561 learned_addr learned my addr 172.21.15.64:0/2559259561 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:25.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c3e34640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30bc1440d0 0x7f30bc1fb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f92e0b37640 1 Processor -- start 2024-03-07T07:19:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f6bb4a25640 1 Processor -- start 2024-03-07T07:19:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f6bb4a25640 1 -- start start 2024-03-07T07:19:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f6bb4a25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba80e42d0 0x7f6ba804cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f6bb4a25640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dc050 0x7f6ba80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f6bb4a25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba80dc560 0x7f6ba80e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:25.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f6bb4a25640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8008b10 con 0x7f6ba80e42d0 2024-03-07T07:19:25.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f6bb4a25640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8056b50 con 0x7f6ba80dc560 2024-03-07T07:19:25.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f6bb4a25640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80e4910 con 0x7f6ba80dc050 2024-03-07T07:19:25.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f30c3e34640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0cbd30 con 0x7f30bc162960 2024-03-07T07:19:25.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f92e0b37640 1 -- start start 2024-03-07T07:19:25.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7ff7c18d0640 1 -- start start 2024-03-07T07:19:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f30c23d1640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30bc1440d0 0x7f30bc1fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f30c3e34640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0cbbb0 con 0x7f30bc1440d0 2024-03-07T07:19:26.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7c18d0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7b403e870 0x7ff7b4188570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f6baffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba80dc560 0x7f6ba80e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f6baffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba80dc560 0x7f6ba80e26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34312/0 (socket says 172.21.15.64:34312) 2024-03-07T07:19:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f6baffff640 1 -- 172.21.15.64:0/1732785841 learned_addr learned my addr 172.21.15.64:0/1732785841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f6baeffd640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dc050 0x7f6ba80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.960+0000 7f8d3fb78640 1 -- start start 2024-03-07T07:19:26.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.959+0000 7f2649a12640 1 -- start start 2024-03-07T07:19:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7c18d0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2649a12640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c0039f0 0x7f26441575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f6baffff640 1 -- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dc050 msgr2=0x7f6ba80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f30c3e34640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0cbeb0 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7c18d0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b40e30f0 0x7ff7b403a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7b403e870 0x7ff7b4188570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7b403e870 0x7ff7b4188570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34326/0 (socket says 172.21.15.64:34326) 2024-03-07T07:19:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f70dd7fa640 1 -- 172.21.15.64:0/2932783724 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4004030 con 0x7f70d80df220 2024-03-07T07:19:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f70dd7fa640 1 -- 172.21.15.64:0/2932783724 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d40041d0 con 0x7f70d80df220 2024-03-07T07:19:26.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d8049d30 con 0x7f70d80df220 2024-03-07T07:19:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7baffd640 1 -- 172.21.15.64:0/2172954735 learned_addr learned my addr 172.21.15.64:0/2172954735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2649a12640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f26440cb670 0x7f26440cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2648a10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c0039f0 0x7f26441575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2648a10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c0039f0 0x7f26441575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34334/0 (socket says 172.21.15.64:34334) 2024-03-07T07:19:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2648a10640 1 -- 172.21.15.64:0/795095423 learned_addr learned my addr 172.21.15.64:0/795095423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2649a12640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26440aa6c0 0x7f26440aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2643fff640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f26440cb670 0x7f26440cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d804a2f0 con 0x7f70d80df220 2024-03-07T07:19:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440ce180 con 0x7f26440aa6c0 2024-03-07T07:19:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264406e2c0 con 0x7f262c0039f0 2024-03-07T07:19:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26440ce300 con 0x7f26440cb670 2024-03-07T07:19:26.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f6baffff640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dc050 0x7f6ba80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f8d3fb78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d300e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f8d3fb78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d300df250 0x7f8d300d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f8d3fb78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d300e2db0 0x7f8d3004d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f8d3fb78640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300065f0 con 0x7f8d300e2db0 2024-03-07T07:19:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f8d3fb78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30006310 con 0x7f8d300df250 2024-03-07T07:19:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f8d3fb78640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300df630 con 0x7f8d30006b10 2024-03-07T07:19:26.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f2649211640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26440aa6c0 0x7f26440aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f51438dd640 1 Processor -- start 2024-03-07T07:19:26.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f8d3d113640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d300df250 0x7f8d300d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f51438dd640 1 -- start start 2024-03-07T07:19:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92e0b37640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4006a50 0x7f92d40db410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f8d3d113640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d300df250 0x7f8d300d6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34344/0 (socket says 172.21.15.64:34344) 2024-03-07T07:19:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4006e40 con 0x7ff7b4054160 2024-03-07T07:19:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40e34d0 con 0x7ff7b403e870 2024-03-07T07:19:26.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4006fc0 con 0x7ff7b40e30f0 2024-03-07T07:19:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6baffff640 1 -- 172.21.15.64:0/1732785841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba80e42d0 msgr2=0x7f6ba804cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6baf7fe640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba80e42d0 0x7f6ba804cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7bb7fe640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b40e30f0 0x7ff7b403a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7ff7ba7fc640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92e0b37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4034020 0x7f92d40e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6baffff640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba80e42d0 0x7f6ba804cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6baffff640 1 -- 172.21.15.64:0/1732785841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8051630 con 0x7f6ba80dc560 2024-03-07T07:19:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f2648a10640 1 -- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f26440cb670 msgr2=0x7f26440cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f2648a10640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f26440cb670 0x7f26440cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f2648a10640 1 -- 172.21.15.64:0/795095423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26440aa6c0 msgr2=0x7f26440aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f2648a10640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26440aa6c0 0x7f26440aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f2648a10640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440ab080 con 0x7f262c0039f0 2024-03-07T07:19:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4006a50 0x7f92d40db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4006a50 0x7f92d40db410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34552/0 (socket says 172.21.15.64:34552) 2024-03-07T07:19:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f8d3d113640 1 -- 172.21.15.64:0/607519820 learned_addr learned my addr 172.21.15.64:0/607519820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f8d3d914640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d300e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6baeffd640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dc050 0x7f6ba80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:26.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6baf7fe640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba80e42d0 0x7f6ba804cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6baffff640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba80dc560 0x7f6ba80e26d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c740 tx=0x7f6ba400cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f2648a10640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c0039f0 0x7f26441575b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26380099c0 tx=0x7f2638004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f30c1bd0640 1 -- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30bc1440d0 msgr2=0x7f30bc1fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.961+0000 7f30c1bd0640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30bc1440d0 0x7f30bc1fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f30c1bd0640 1 -- 172.21.15.64:0/2559259561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc162960 msgr2=0x7f30bc141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f30c1bd0640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc162960 0x7f30bc141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f30c1bd0640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc142db0 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f30c1bd0640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc0cf6d0 0x7f30bc1f5460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30b800d800 tx=0x7f30b800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92e0b37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d404b3c0 0x7f92d417d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92e0b37640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d404b7a0 con 0x7f92d4006a50 2024-03-07T07:19:26.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92e0b37640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4006530 con 0x7f92d404b3c0 2024-03-07T07:19:26.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92e0b37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4054830 con 0x7f92d4034020 2024-03-07T07:19:26.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92daffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4034020 0x7f92d40e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7ff7baffd640 1 -- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b40e30f0 msgr2=0x7ff7b403a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7ff7baffd640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b40e30f0 0x7ff7b403a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7ff7baffd640 1 -- 172.21.15.64:0/2172954735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 msgr2=0x7ff7b4039b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7ff7baffd640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4039b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f30b27fc640 1 -- 172.21.15.64:0/2559259561 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8018030 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7ff7baffd640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4052440 con 0x7ff7b403e870 2024-03-07T07:19:26.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f70dd7fa640 1 -- 172.21.15.64:0/2932783724 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4014620 con 0x7f70d80df220 2024-03-07T07:19:26.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7ff7bb7fe640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b40e30f0 0x7ff7b403a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:26.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6b9bfff640 1 -- 172.21.15.64:0/1732785841 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4018070 con 0x7f6ba80dc560 2024-03-07T07:19:26.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f30c3e34640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc143030 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7ff7ba7fc640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4039b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:26.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f6b9bfff640 1 -- 172.21.15.64:0/1732785841 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba40072d0 con 0x7f6ba80dc560 2024-03-07T07:19:26.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92daffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4034020 0x7f92d40e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:57050/0 (socket says 172.21.15.64:57050) 2024-03-07T07:19:26.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f30c3e34640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc1435f0 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.962+0000 7f92db7fe640 1 -- 172.21.15.64:0/2753624805 learned_addr learned my addr 172.21.15.64:0/2753624805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f92db7fe640 1 -- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4034020 msgr2=0x7f92d40e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f92db7fe640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4034020 0x7f92d40e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f92db7fe640 1 -- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d404b3c0 msgr2=0x7f92d417d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f92db7fe640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d404b3c0 0x7f92d417d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f92db7fe640 1 -- 172.21.15.64:0/2753624805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40e77b0 con 0x7f92d4006a50 2024-03-07T07:19:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f26417fa640 1 -- 172.21.15.64:0/795095423 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263800f000 con 0x7f262c0039f0 2024-03-07T07:19:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f26417fa640 1 -- 172.21.15.64:0/795095423 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263802f430 con 0x7f262c0039f0 2024-03-07T07:19:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f26417fa640 1 -- 172.21.15.64:0/795095423 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638004020 con 0x7f262c0039f0 2024-03-07T07:19:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f8d3e115640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d300e2db0 0x7f8d3004d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f6bb4a25640 1 -- 172.21.15.64:0/1732785841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba80518f0 con 0x7f6ba80dc560 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26441f86d0 con 0x7f262c0039f0 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26441f8c90 con 0x7f262c0039f0 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f51438dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c06de60 0x7f513c0a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f51438dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513c0722e0 0x7f513c144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f51438dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f513c0cf140 0x7f513c152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f51438dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b11e0 con 0x7f513c0722e0 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f51438dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0cfc30 con 0x7f513c06de60 2024-03-07T07:19:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f51438dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b1360 con 0x7f513c0cf140 2024-03-07T07:19:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f8d3d113640 1 -- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 msgr2=0x7f8d300e1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f8d3d113640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d300e1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f8d3d113640 1 -- 172.21.15.64:0/607519820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d300e2db0 msgr2=0x7f8d3004d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f8d3d113640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d300e2db0 0x7f8d3004d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f8d3d113640 1 -- 172.21.15.64:0/607519820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3004cf50 con 0x7f8d300df250 2024-03-07T07:19:26.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.963+0000 7f6bb4a25640 1 -- 172.21.15.64:0/1732785841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8051eb0 con 0x7f6ba80dc560 2024-03-07T07:19:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f30b27fc640 1 -- 172.21.15.64:0/2559259561 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b80096a0 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513c0722e0 0x7f513c144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513c0722e0 0x7f513c144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34568/0 (socket says 172.21.15.64:34568) 2024-03-07T07:19:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 -- 172.21.15.64:0/2594714221 learned_addr learned my addr 172.21.15.64:0/2594714221 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f6b9bfff640 1 -- 172.21.15.64:0/1732785841 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40047e0 con 0x7f6ba80dc560 2024-03-07T07:19:26.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f30b27fc640 1 -- 172.21.15.64:0/2559259561 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8009870 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5141679640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c06de60 0x7f513c0a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5141e7a640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f513c0cf140 0x7f513c152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d8005390 con 0x7f70d80df220 2024-03-07T07:19:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 -- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f513c0cf140 msgr2=0x7f513c152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f513c0cf140 0x7f513c152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 -- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c06de60 msgr2=0x7f513c0a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c06de60 0x7f513c0a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f5140e78640 1 -- 172.21.15.64:0/2594714221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c1451d0 con 0x7f513c0722e0 2024-03-07T07:19:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.964+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b88005290 con 0x7f6ba80dc560 2024-03-07T07:19:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7ff7baffd640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7b403e870 0x7ff7b4188570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c3c0 tx=0x7ff7ac00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f92daffd640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4034020 0x7f92d40e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-07T07:19:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f5140e78640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513c0722e0 0x7f513c144c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5134002010 tx=0x7f513400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7ff7a3fff640 1 -- 172.21.15.64:0/2172954735 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac017070 con 0x7ff7b403e870 2024-03-07T07:19:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7ff7a3fff640 1 -- 172.21.15.64:0/2172954735 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac005ce0 con 0x7ff7b403e870 2024-03-07T07:19:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f70dd7fa640 1 -- 172.21.15.64:0/2932783724 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f70d40147c0 con 0x7f70d80df220 2024-03-07T07:19:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc062c90 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f92db7fe640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4006a50 0x7f92d40db410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92d000b440 tx=0x7f92d000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f8d3d113640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d300df250 0x7f8d300d6a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d38050ba0 tx=0x7f8d38074e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f30b27fc640 1 -- 172.21.15.64:0/2559259561 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f30b801b070 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2644062c90 con 0x7f262c0039f0 2024-03-07T07:19:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f70dd7fa640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f70c406b5a0 0x7f70c406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f70dd7fa640 1 -- 172.21.15.64:0/2932783724 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70d408a820 con 0x7f70d80df220 2024-03-07T07:19:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f70e4bd0640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f70c406b5a0 0x7f70c406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f30b27fc640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f30a006b520 0x7f30a006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f5131ffb640 1 -- 172.21.15.64:0/2594714221 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134016020 con 0x7f513c0722e0 2024-03-07T07:19:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f5131ffb640 1 -- 172.21.15.64:0/2594714221 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134012420 con 0x7f513c0722e0 2024-03-07T07:19:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f92c7fff640 1 -- 172.21.15.64:0/2753624805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0015020 con 0x7f92d4006a50 2024-03-07T07:19:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f92c7fff640 1 -- 172.21.15.64:0/2753624805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d0011890 con 0x7f92d4006a50 2024-03-07T07:19:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f92c7fff640 1 -- 172.21.15.64:0/2753624805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0011c80 con 0x7f92d4006a50 2024-03-07T07:19:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.965+0000 7f92e0b37640 1 -- 172.21.15.64:0/2753624805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d40e7a50 con 0x7f92d4006a50 2024-03-07T07:19:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f92e0b37640 1 -- 172.21.15.64:0/2753624805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d40e8010 con 0x7f92d4006a50 2024-03-07T07:19:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f6b9bfff640 1 -- 172.21.15.64:0/1732785841 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6ba401b070 con 0x7f6ba80dc560 2024-03-07T07:19:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f30b27fc640 1 -- 172.21.15.64:0/2559259561 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30b808a510 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7ff7a3fff640 1 -- 172.21.15.64:0/2172954735 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0072d0 con 0x7ff7b403e870 2024-03-07T07:19:26.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b40513a0 con 0x7ff7b403e870 2024-03-07T07:19:26.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b4051960 con 0x7ff7b403e870 2024-03-07T07:19:26.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f6b9bfff640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6b9406b5a0 0x7f6b9406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f6b9bfff640 1 -- 172.21.15.64:0/1732785841 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ba4089750 con 0x7f6ba80dc560 2024-03-07T07:19:26.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f5131ffb640 1 -- 172.21.15.64:0/2594714221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134012810 con 0x7f513c0722e0 2024-03-07T07:19:26.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f30c13cf640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f30a006b520 0x7f30a006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f26417fa640 1 -- 172.21.15.64:0/795095423 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2638049030 con 0x7f262c0039f0 2024-03-07T07:19:26.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.966+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c1533c0 con 0x7f513c0722e0 2024-03-07T07:19:26.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f6baf7fe640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6b9406b5a0 0x7f6b9406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f70e4bd0640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f70c406b5a0 0x7f70c406da60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70e0067e90 tx=0x7f70e0073820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f30c13cf640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f30a006b520 0x7f30a006d9e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30bc157e50 tx=0x7f30ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f26417fa640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f263006b5a0 0x7f263006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c153940 con 0x7f513c0722e0 2024-03-07T07:19:26.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f26417fa640 1 -- 172.21.15.64:0/795095423 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26380ac2a0 con 0x7f262c0039f0 2024-03-07T07:19:26.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f6baf7fe640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6b9406b5a0 0x7f6b9406da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c036000 tx=0x7f6b9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f2643fff640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f263006b5a0 0x7f263006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7f92e0b37640 1 -- 172.21.15.64:0/2753624805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d40057e0 con 0x7f92d4006a50 2024-03-07T07:19:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.967+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4005810 con 0x7ff7b403e870 2024-03-07T07:19:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.968+0000 7f2643fff640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f263006b5a0 0x7f263006da60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f263c0070b0 tx=0x7f263c009870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.968+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c062c90 con 0x7f513c0722e0 2024-03-07T07:19:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.968+0000 7f92c7fff640 1 -- 172.21.15.64:0/2753624805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f92d0018020 con 0x7f92d4006a50 2024-03-07T07:19:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f92c7fff640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f92c006b5d0 0x7f92c006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f5131ffb640 1 -- 172.21.15.64:0/2594714221 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5134019020 con 0x7f513c0722e0 2024-03-07T07:19:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f92c7fff640 1 -- 172.21.15.64:0/2753624805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92d008a890 con 0x7f92d4006a50 2024-03-07T07:19:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f92daffd640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f92c006b5d0 0x7f92c006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f5131ffb640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f512006b520 0x7f512006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f5131ffb640 1 -- 172.21.15.64:0/2594714221 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f513408a6b0 con 0x7f513c0722e0 2024-03-07T07:19:26.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f5141679640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f512006b520 0x7f512006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.969+0000 7f92daffd640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f92c006b5d0 0x7f92c006da90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f92c8034000 tx=0x7f92c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.970+0000 7f5141679640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f512006b520 0x7f512006d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5138036000 tx=0x7f5138008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.972+0000 7f92c7fff640 1 -- 172.21.15.64:0/2753624805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f92d000d040 con 0x7f92d4006a50 2024-03-07T07:19:26.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.974+0000 7f8d2e7fc640 1 -- 172.21.15.64:0/607519820 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380747e0 con 0x7f8d300df250 2024-03-07T07:19:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.975+0000 7f70dd7fa640 1 -- 172.21.15.64:0/2932783724 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70d4011340 con 0x7f70d80df220 2024-03-07T07:19:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.975+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3004cc70 con 0x7f8d300df250 2024-03-07T07:19:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.975+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d300d6560 con 0x7f8d300df250 2024-03-07T07:19:26.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.976+0000 7f30b27fc640 1 -- 172.21.15.64:0/2559259561 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f30bc142db0 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.976+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d30008e50 con 0x7f8d300df250 2024-03-07T07:19:26.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.977+0000 7f5131ffb640 1 -- 172.21.15.64:0/2594714221 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5134020030 con 0x7f513c0722e0 2024-03-07T07:19:26.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.980+0000 7f6b9bfff640 1 -- 172.21.15.64:0/1732785841 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ba4057f60 con 0x7f6ba80dc560 2024-03-07T07:19:26.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.983+0000 7ff7a3fff640 1 -- 172.21.15.64:0/2172954735 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff7ac01a070 con 0x7ff7b403e870 2024-03-07T07:19:26.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.983+0000 7ff7a3fff640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff7a806b4d0 0x7ff7a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.984+0000 7f8d2e7fc640 1 -- 172.21.15.64:0/607519820 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d380008d0 con 0x7f8d300df250 2024-03-07T07:19:26.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.984+0000 7f8d2e7fc640 1 -- 172.21.15.64:0/607519820 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3809a060 con 0x7f8d300df250 2024-03-07T07:19:26.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.984+0000 7ff7a3fff640 1 -- 172.21.15.64:0/2172954735 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7ac056170 con 0x7ff7b403e870 2024-03-07T07:19:26.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.984+0000 7ff7ba7fc640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff7a806b4d0 0x7ff7a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.984+0000 7f26417fa640 1 -- 172.21.15.64:0/795095423 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2638050020 con 0x7f262c0039f0 2024-03-07T07:19:26.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.985+0000 7ff7ba7fc640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff7a806b4d0 0x7ff7a806d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40df3b0 tx=0x7ff7b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.985+0000 7f8d2e7fc640 1 -- 172.21.15.64:0/607519820 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8d380b4820 con 0x7f8d300df250 2024-03-07T07:19:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.986+0000 7f8d2e7fc640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f8d1c06b4d0 0x7f8d1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.986+0000 7f8d3d914640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f8d1c06b4d0 0x7f8d1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.986+0000 7f8d3d914640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f8d1c06b4d0 0x7f8d1c06d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d34001780 tx=0x7f8d34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.988+0000 7f8d2e7fc640 1 -- 172.21.15.64:0/607519820 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d380d19e0 con 0x7f8d300df250 2024-03-07T07:19:26.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.988+0000 7ff7a3fff640 1 -- 172.21.15.64:0/2172954735 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7ac0580d0 con 0x7ff7b403e870 2024-03-07T07:19:26.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:25.993+0000 7f8d2e7fc640 1 -- 172.21.15.64:0/607519820 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8d380d3940 con 0x7f8d300df250 2024-03-07T07:19:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.375+0000 7f92e0b37640 1 -- 172.21.15.64:0/2753624805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f92d404a960 con 0x7f92d4006a50 2024-03-07T07:19:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.375+0000 7f92c7fff640 1 -- 172.21.15.64:0/2753624805 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f92d0011e20 con 0x7f92d4006a50 2024-03-07T07:19:26.375 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 -- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f92c006b5d0 msgr2=0x7f92c006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f92c006b5d0 0x7f92c006da90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f92c8034000 tx=0x7f92c8008030 comp rx=0 tx=0).stop 2024-03-07T07:19:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 -- 172.21.15.64:0/2753624805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4006a50 msgr2=0x7f92d40db410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4006a50 0x7f92d40db410 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92d000b440 tx=0x7f92d000b900 comp rx=0 tx=0).stop 2024-03-07T07:19:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 -- 172.21.15.64:0/2753624805 shutdown_connections 2024-03-07T07:19:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f92c006b5d0 0x7f92c006da90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92d404b3c0 0x7f92d417d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f92d4034020 0x7f92d40e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.379+0000 7f92c5ffb640 1 --2- 172.21.15.64:0/2753624805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92d4006a50 0x7f92d40db410 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.380+0000 7f92c5ffb640 1 -- 172.21.15.64:0/2753624805 >> 172.21.15.64:0/2753624805 conn(0x7f92d400a400 msgr2=0x7f92d40361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.380+0000 7f92c5ffb640 1 -- 172.21.15.64:0/2753624805 shutdown_connections 2024-03-07T07:19:26.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.380+0000 7f92c5ffb640 1 -- 172.21.15.64:0/2753624805 wait complete. 2024-03-07T07:19:26.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.395+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f70d80362f0 con 0x7f70d80df220 2024-03-07T07:19:26.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.395+0000 7f70dd7fa640 1 -- 172.21.15.64:0/2932783724 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70d4059710 con 0x7f70d80df220 2024-03-07T07:19:26.395 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-03-07T07:19:26.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-03-07T07:19:26.397 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.398+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f70c406b5a0 msgr2=0x7f70c406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.398+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f70c406b5a0 0x7f70c406da60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70e0067e90 tx=0x7f70e0073820 comp rx=0 tx=0).stop 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.398+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d80df220 msgr2=0x7f70d8175030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.398+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d80df220 0x7f70d8175030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f70d400d5d0 tx=0x7f70d400da90 comp rx=0 tx=0).stop 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.398+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 shutdown_connections 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.398+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f70c406b5a0 0x7f70c406da60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.399+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f70d8053c70 0x7f70d80e2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.399+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70d80df220 0x7f70d8175030 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.399+0000 7f70e6e34640 1 --2- 172.21.15.64:0/2932783724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70d8033ef0 0x7f70d80e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.399+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 >> 172.21.15.64:0/2932783724 conn(0x7f70d8009fb0 msgr2=0x7f70d80d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.399+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 shutdown_connections 2024-03-07T07:19:26.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.399+0000 7f70e6e34640 1 -- 172.21.15.64:0/2932783724 wait complete. 2024-03-07T07:19:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.402+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8d30005810 con 0x7f8d300df250 2024-03-07T07:19:26.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.402+0000 7f8d2e7fc640 1 -- 172.21.15.64:0/607519820 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d380d79e0 con 0x7f8d300df250 2024-03-07T07:19:26.402 INFO:teuthology.orchestra.run.smithi064.stdout:55834574855 2024-03-07T07:19:26.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.404+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f513c072da0 con 0x7f513c0722e0 2024-03-07T07:19:26.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.404+0000 7f5131ffb640 1 -- 172.21.15.64:0/2594714221 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f513c1451d0 con 0x7f513c0722e0 2024-03-07T07:19:26.404 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f8d1c06b4d0 msgr2=0x7f8d1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f8d1c06b4d0 0x7f8d1c06d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d34001780 tx=0x7f8d34009040 comp rx=0 tx=0).stop 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d300df250 msgr2=0x7f8d300d6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d300df250 0x7f8d300d6a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d38050ba0 tx=0x7f8d38074e70 comp rx=0 tx=0).stop 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 shutdown_connections 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f8d1c06b4d0 0x7f8d1c06d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d300e2db0 0x7f8d3004d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d300df250 0x7f8d300d6a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.405+0000 7f8d3fb78640 1 --2- 172.21.15.64:0/607519820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d30006b10 0x7f8d300e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.406+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 >> 172.21.15.64:0/607519820 conn(0x7f8d3000a430 msgr2=0x7f8d3004f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.406+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 shutdown_connections 2024-03-07T07:19:26.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.406+0000 7f8d3fb78640 1 -- 172.21.15.64:0/607519820 wait complete. 2024-03-07T07:19:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f512006b520 msgr2=0x7f512006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f512006b520 0x7f512006d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5138036000 tx=0x7f5138008030 comp rx=0 tx=0).stop 2024-03-07T07:19:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513c0722e0 msgr2=0x7f513c144c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513c0722e0 0x7f513c144c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5134002010 tx=0x7f513400a430 comp rx=0 tx=0).stop 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 shutdown_connections 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f512006b520 0x7f512006d9e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513c0722e0 0x7f513c144c90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f513c0cf140 0x7f513c152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 --2- 172.21.15.64:0/2594714221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f513c06de60 0x7f513c0a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 >> 172.21.15.64:0/2594714221 conn(0x7f513c0b1d00 msgr2=0x7f513c0b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 shutdown_connections 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f51438dd640 1 -- 172.21.15.64:0/2594714221 wait complete. 2024-03-07T07:19:26.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6b88005580 con 0x7f6ba80dc560 2024-03-07T07:19:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2644142b60 con 0x7f262c0039f0 2024-03-07T07:19:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.408+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f30bc0cc310 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.409+0000 7f6b9bfff640 1 -- 172.21.15.64:0/1732785841 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ba405c000 con 0x7f6ba80dc560 2024-03-07T07:19:26.411 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.409+0000 7f30b27fc640 1 -- 172.21.15.64:0/2559259561 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30b801be40 con 0x7f30bc0cf6d0 2024-03-07T07:19:26.411 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:26.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.409+0000 7f26417fa640 1 -- 172.21.15.64:0/795095423 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f263803d040 con 0x7f262c0039f0 2024-03-07T07:19:26.412 INFO:teuthology.orchestra.run.smithi064.stdout:55834574856 2024-03-07T07:19:26.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6b9406b5a0 msgr2=0x7f6b9406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6b9406b5a0 0x7f6b9406da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c036000 tx=0x7f6b9c008030 comp rx=0 tx=0).stop 2024-03-07T07:19:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba80dc560 msgr2=0x7f6ba80e26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba80dc560 0x7f6ba80e26d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c740 tx=0x7f6ba400cc00 comp rx=0 tx=0).stop 2024-03-07T07:19:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 shutdown_connections 2024-03-07T07:19:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f30a006b520 msgr2=0x7f30a006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6b9406b5a0 0x7f6b9406da60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ba80dc560 0x7f6ba80e26d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f309ffff640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f30a006b520 0x7f30a006d9e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30bc157e50 tx=0x7f30ac008040 comp rx=0 tx=0).stop 2024-03-07T07:19:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ba80dc050 0x7f6ba80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc0cf6d0 msgr2=0x7f30bc1f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 --2- 172.21.15.64:0/1732785841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ba80e42d0 0x7f6ba804cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f309ffff640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc0cf6d0 0x7f30bc1f5460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30b800d800 tx=0x7f30b800dcc0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.412+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 >> 172.21.15.64:0/1732785841 conn(0x7f6ba801ef40 msgr2=0x7f6ba80de800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 shutdown_connections 2024-03-07T07:19:26.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f30a006b520 0x7f30a006d9e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30bc162960 0x7f30bc141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30bc1440d0 0x7f30bc1fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 --2- 172.21.15.64:0/2559259561 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30bc0cf6d0 0x7f30bc1f5460 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 >> 172.21.15.64:0/2559259561 conn(0x7f30bc0cfb90 msgr2=0x7f30bc0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f263006b5a0 msgr2=0x7f263006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 shutdown_connections 2024-03-07T07:19:26.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f263006b5a0 0x7f263006da60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f263c0070b0 tx=0x7f263c009870 comp rx=0 tx=0).stop 2024-03-07T07:19:26.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f6b99ffb640 1 -- 172.21.15.64:0/1732785841 wait complete. 2024-03-07T07:19:26.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c0039f0 msgr2=0x7f26441575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c0039f0 0x7f26441575b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f26380099c0 tx=0x7f2638004730 comp rx=0 tx=0).stop 2024-03-07T07:19:26.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 shutdown_connections 2024-03-07T07:19:26.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f309ffff640 1 -- 172.21.15.64:0/2559259561 wait complete. 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 shutdown_connections 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f263006b5a0 0x7f263006da60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26440aa6c0 0x7f26440aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f26440cb670 0x7f26440cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 --2- 172.21.15.64:0/795095423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f262c0039f0 0x7f26441575b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 >> 172.21.15.64:0/795095423 conn(0x7f2644067090 msgr2=0x7f2644070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 shutdown_connections 2024-03-07T07:19:26.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.413+0000 7f2649a12640 1 -- 172.21.15.64:0/795095423 wait complete. 2024-03-07T07:19:26.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.416+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff7b4188ab0 con 0x7ff7b403e870 2024-03-07T07:19:26.420 INFO:teuthology.orchestra.run.smithi064.stdout:55834574854 2024-03-07T07:19:26.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.416+0000 7ff7a3fff640 1 -- 172.21.15.64:0/2172954735 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7ac05c170 con 0x7ff7b403e870 2024-03-07T07:19:26.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-03-07T07:19:26.421 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff7a806b4d0 msgr2=0x7ff7a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff7a806b4d0 0x7ff7a806d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40df3b0 tx=0x7ff7b0008040 comp rx=0 tx=0).stop 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7b403e870 msgr2=0x7ff7b4188570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7b403e870 0x7ff7b4188570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c3c0 tx=0x7ff7ac00c880 comp rx=0 tx=0).stop 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 shutdown_connections 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff7a806b4d0 0x7ff7a806d990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7b4054160 0x7ff7b4039b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7b40e30f0 0x7ff7b403a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 --2- 172.21.15.64:0/2172954735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7b403e870 0x7ff7b4188570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 >> 172.21.15.64:0/2172954735 conn(0x7ff7b4055a10 msgr2=0x7ff7b4038110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 shutdown_connections 2024-03-07T07:19:26.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.419+0000 7ff7c18d0640 1 -- 172.21.15.64:0/2172954735 wait complete. 2024-03-07T07:19:26.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-03-07T07:19:26.429 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-03-07T07:19:26.430 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-03-07T07:19:26.431 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-03-07T07:19:26.434 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-03-07T07:19:26.435 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-03-07T07:19:26.435 DEBUG:teuthology.parallel:result is None 2024-03-07T07:19:26.436 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-07T07:19:26.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:19:26.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.575+0000 7fa7c7999640 1 Processor -- start 2024-03-07T07:19:26.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c7999640 1 -- start start 2024-03-07T07:19:26.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c7999640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa7c0147270 0x7fa7c006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c7999640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147810 0x7fa7c006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c7999640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c00d1e70 con 0x7fa7c00d2c10 2024-03-07T07:19:26.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c7999640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c0147650 con 0x7fa7c0147270 2024-03-07T07:19:26.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c7999640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00b7520 con 0x7fa7c0147810 2024-03-07T07:19:26.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c4f34640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa7c00d2c10 0x7fa7c0162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48724/0 (socket says 172.21.15.64:48724) 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.576+0000 7fa7c4f34640 1 -- 172.21.15.64:0/3612555450 learned_addr learned my addr 172.21.15.64:0/3612555450 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7c5f36640 1 --2- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147810 0x7fa7c006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7c5735640 1 --2- 172.21.15.64:0/3612555450 >> v2:172.21.15.120:3300/0 conn(0x7fa7c0147270 0x7fa7c006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7bffff640 1 -- 172.21.15.64:0/3612555450 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145255374 0 0) 0x7fa7c00d1e70 con 0x7fa7c00d2c10 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7bffff640 1 -- 172.21.15.64:0/3612555450 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a8003520 con 0x7fa7c00d2c10 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7bffff640 1 -- 172.21.15.64:0/3612555450 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3267760168 0 0) 0x7fa7a8003520 con 0x7fa7c00d2c10 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7c5735640 1 -- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147810 msgr2=0x7fa7c006ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7c5735640 1 --2- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147810 0x7fa7c006ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7c5735640 1 -- 172.21.15.64:0/3612555450 >> v1:172.21.15.64:6789/0 conn(0x7fa7c00d2c10 legacy=0x7fa7c0162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7c5735640 1 -- 172.21.15.64:0/3612555450 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00d2130 con 0x7fa7c0147270 2024-03-07T07:19:26.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.577+0000 7fa7c5f36640 1 --2- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147810 0x7fa7c006ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7c5735640 1 --2- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c0147270 0x7fa7c006b510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009d90 tx=0x7fa7b802bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669524a6f8dfbebe server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7bffff640 1 -- 172.21.15.64:0/3612555450 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8039070 con 0x7fa7c0147270 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7bffff640 1 -- 172.21.15.64:0/3612555450 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b802c810 con 0x7fa7c0147270 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7bffff640 1 -- 172.21.15.64:0/3612555450 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4001690 con 0x7fa7c0147270 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7c7999640 1 -- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c0147270 msgr2=0x7fa7c006b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7c7999640 1 --2- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c0147270 0x7fa7c006b510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009d90 tx=0x7fa7b802bce0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7c7999640 1 -- 172.21.15.64:0/3612555450 shutdown_connections 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7c7999640 1 --2- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147810 0x7fa7c006ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7c7999640 1 --2- 172.21.15.64:0/3612555450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c0147270 0x7fa7c006b510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.578+0000 7fa7c7999640 1 -- 172.21.15.64:0/3612555450 >> 172.21.15.64:0/3612555450 conn(0x7fa7c006d4c0 msgr2=0x7fa7c0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.579+0000 7fa7c7999640 1 -- 172.21.15.64:0/3612555450 shutdown_connections 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.579+0000 7fa7c7999640 1 -- 172.21.15.64:0/3612555450 wait complete. 2024-03-07T07:19:26.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 Processor -- start 2024-03-07T07:19:26.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 -- start start 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c00d2c10 0x7fa7c015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147270 0x7fa7c00cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7c0147810 0x7fa7c00ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00b71d0 con 0x7fa7c0147810 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00d26f0 con 0x7fa7c00d2c10 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c7999640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0147d60 con 0x7fa7c0147270 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c5f36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7c0147810 0x7fa7c00ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c5f36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7c0147810 0x7fa7c00ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34574/0 (socket says 172.21.15.64:34574) 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c5f36640 1 -- 172.21.15.64:0/501357137 learned_addr learned my addr 172.21.15.64:0/501357137 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c5735640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c00d2c10 0x7fa7c015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.580+0000 7fa7c4f34640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147270 0x7fa7c00cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c5f36640 1 -- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147270 msgr2=0x7fa7c00cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c5f36640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147270 0x7fa7c00cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c5f36640 1 -- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c00d2c10 msgr2=0x7fa7c015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c5f36640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c00d2c10 0x7fa7c015dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c5f36640 1 -- 172.21.15.64:0/501357137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00cc410 con 0x7fa7c0147810 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c5f36640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7c0147810 0x7fa7c00ced00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e880 tx=0x7fa7b000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7bdffb640 1 -- 172.21.15.64:0/501357137 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0017070 con 0x7fa7c0147810 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7bdffb640 1 -- 172.21.15.64:0/501357137 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0004020 con 0x7fa7c0147810 2024-03-07T07:19:26.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c015e6a0 con 0x7fa7c0147810 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7bdffb640 1 -- 172.21.15.64:0/501357137 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00043f0 con 0x7fa7c0147810 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.581+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c015ec60 con 0x7fa7c0147810 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.582+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c0062c90 con 0x7fa7c0147810 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.582+0000 7fa7bdffb640 1 -- 172.21.15.64:0/501357137 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa7b000c940 con 0x7fa7c0147810 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.583+0000 7fa7bdffb640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa7ac06b500 0x7fa7ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.583+0000 7fa7bdffb640 1 -- 172.21.15.64:0/501357137 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7b0089e70 con 0x7fa7c0147810 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.583+0000 7fa7c5735640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa7ac06b500 0x7fa7ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.583+0000 7fa7c5735640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa7ac06b500 0x7fa7ac06d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8036000 tx=0x7fa7b8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.587+0000 7fa7bdffb640 1 -- 172.21.15.64:0/501357137 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7b0058730 con 0x7fa7c0147810 2024-03-07T07:19:26.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.768+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7c00d1530 con 0x7fa7ac06b500 2024-03-07T07:19:26.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.774+0000 7fa7bdffb640 1 -- 172.21.15.64:0/501357137 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7fa7c00d1530 con 0x7fa7ac06b500 2024-03-07T07:19:26.777 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:26.778 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa7ac06b500 msgr2=0x7fa7ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa7ac06b500 0x7fa7ac06d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8036000 tx=0x7fa7b8030000 comp rx=0 tx=0).stop 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7c0147810 msgr2=0x7fa7c00ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7c0147810 0x7fa7c00ced00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e880 tx=0x7fa7b000ed40 comp rx=0 tx=0).stop 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 shutdown_connections 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa7ac06b500 0x7fa7ac06d9c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7c0147810 0x7fa7c00ced00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa7c0147270 0x7fa7c00cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 --2- 172.21.15.64:0/501357137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7c00d2c10 0x7fa7c015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.777+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 >> 172.21.15.64:0/501357137 conn(0x7fa7c006d4c0 msgr2=0x7fa7c0146430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.778+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 shutdown_connections 2024-03-07T07:19:26.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.778+0000 7fa7c7999640 1 -- 172.21.15.64:0/501357137 wait complete. 2024-03-07T07:19:26.788 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-07T07:19:26.181519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.894564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069031+0000","last_change":"2024-03-07T07:19:16.302696+0000","last_active":"2024-03-07T07:19:18.069031+0000","last_peered":"2024-03-07T07:19:18.069031+0000","last_clean":"2024-03-07T07:19:18.069031+0000","last_became_active":"2024-03-07T07:19:16.302389+0000","last_became_peered":"2024-03-07T07:19:16.302389+0000","last_unstale":"2024-03-07T07:19:18.069031+0000","last_undegraded":"2024-03-07T07:19:18.069031+0000","last_fullsized":"2024-03-07T07:19:18.069031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294255+0000","last_change":"2024-03-07T07:19:16.303217+0000","last_active":"2024-03-07T07:19:17.294255+0000","last_peered":"2024-03-07T07:19:17.294255+0000","last_clean":"2024-03-07T07:19:17.294255+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:17.294255+0000","last_undegraded":"2024-03-07T07:19:17.294255+0000","last_fullsized":"2024-03-07T07:19:17.294255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870246+0000","last_change":"2024-03-07T07:19:16.292858+0000","last_active":"2024-03-07T07:19:17.870246+0000","last_peered":"2024-03-07T07:19:17.870246+0000","last_clean":"2024-03-07T07:19:17.870246+0000","last_became_active":"2024-03-07T07:19:16.292593+0000","last_became_peered":"2024-03-07T07:19:16.292593+0000","last_unstale":"2024-03-07T07:19:17.870246+0000","last_undegraded":"2024-03-07T07:19:17.870246+0000","last_fullsized":"2024-03-07T07:19:17.870246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417391+0000","last_change":"2024-03-07T07:19:16.310590+0000","last_active":"2024-03-07T07:19:17.417391+0000","last_peered":"2024-03-07T07:19:17.417391+0000","last_clean":"2024-03-07T07:19:17.417391+0000","last_became_active":"2024-03-07T07:19:16.310298+0000","last_became_peered":"2024-03-07T07:19:16.310298+0000","last_unstale":"2024-03-07T07:19:17.417391+0000","last_undegraded":"2024-03-07T07:19:17.417391+0000","last_fullsized":"2024-03-07T07:19:17.417391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069132+0000","last_change":"2024-03-07T07:19:16.299586+0000","last_active":"2024-03-07T07:19:18.069132+0000","last_peered":"2024-03-07T07:19:18.069132+0000","last_clean":"2024-03-07T07:19:18.069132+0000","last_became_active":"2024-03-07T07:19:16.299265+0000","last_became_peered":"2024-03-07T07:19:16.299265+0000","last_unstale":"2024-03-07T07:19:18.069132+0000","last_undegraded":"2024-03-07T07:19:18.069132+0000","last_fullsized":"2024-03-07T07:19:18.069132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417301+0000","last_change":"2024-03-07T07:19:16.312616+0000","last_active":"2024-03-07T07:19:17.417301+0000","last_peered":"2024-03-07T07:19:17.417301+0000","last_clean":"2024-03-07T07:19:17.417301+0000","last_became_active":"2024-03-07T07:19:16.312339+0000","last_became_peered":"2024-03-07T07:19:16.312339+0000","last_unstale":"2024-03-07T07:19:17.417301+0000","last_undegraded":"2024-03-07T07:19:17.417301+0000","last_fullsized":"2024-03-07T07:19:17.417301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068749+0000","last_change":"2024-03-07T07:19:16.302351+0000","last_active":"2024-03-07T07:19:18.068749+0000","last_peered":"2024-03-07T07:19:18.068749+0000","last_clean":"2024-03-07T07:19:18.068749+0000","last_became_active":"2024-03-07T07:19:16.302071+0000","last_became_peered":"2024-03-07T07:19:16.302071+0000","last_unstale":"2024-03-07T07:19:18.068749+0000","last_undegraded":"2024-03-07T07:19:18.068749+0000","last_fullsized":"2024-03-07T07:19:18.068749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009417+0000","last_change":"2024-03-07T07:19:16.310400+0000","last_active":"2024-03-07T07:19:18.009417+0000","last_peered":"2024-03-07T07:19:18.009417+0000","last_clean":"2024-03-07T07:19:18.009417+0000","last_became_active":"2024-03-07T07:19:16.310066+0000","last_became_peered":"2024-03-07T07:19:16.310066+0000","last_unstale":"2024-03-07T07:19:18.009417+0000","last_undegraded":"2024-03-07T07:19:18.009417+0000","last_fullsized":"2024-03-07T07:19:18.009417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069013+0000","last_change":"2024-03-07T07:19:16.307888+0000","last_active":"2024-03-07T07:19:18.069013+0000","last_peered":"2024-03-07T07:19:18.069013+0000","last_clean":"2024-03-07T07:19:18.069013+0000","last_became_active":"2024-03-07T07:19:16.306871+0000","last_became_peered":"2024-03-07T07:19:16.306871+0000","last_unstale":"2024-03-07T07:19:18.069013+0000","last_undegraded":"2024-03-07T07:19:18.069013+0000","last_fullsized":"2024-03-07T07:19:18.069013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296721+0000","last_change":"2024-03-07T07:19:16.306115+0000","last_active":"2024-03-07T07:19:17.296721+0000","last_peered":"2024-03-07T07:19:17.296721+0000","last_clean":"2024-03-07T07:19:17.296721+0000","last_became_active":"2024-03-07T07:19:16.305591+0000","last_became_peered":"2024-03-07T07:19:16.305591+0000","last_unstale":"2024-03-07T07:19:17.296721+0000","last_undegraded":"2024-03-07T07:19:17.296721+0000","last_fullsized":"2024-03-07T07:19:17.296721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296408+0000","last_change":"2024-03-07T07:19:16.305713+0000","last_active":"2024-03-07T07:19:17.296408+0000","last_peered":"2024-03-07T07:19:17.296408+0000","last_clean":"2024-03-07T07:19:17.296408+0000","last_became_active":"2024-03-07T07:19:16.305408+0000","last_became_peered":"2024-03-07T07:19:16.305408+0000","last_unstale":"2024-03-07T07:19:17.296408+0000","last_undegraded":"2024-03-07T07:19:17.296408+0000","last_fullsized":"2024-03-07T07:19:17.296408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297262+0000","last_change":"2024-03-07T07:19:16.301659+0000","last_active":"2024-03-07T07:19:17.297262+0000","last_peered":"2024-03-07T07:19:17.297262+0000","last_clean":"2024-03-07T07:19:17.297262+0000","last_became_active":"2024-03-07T07:19:16.301033+0000","last_became_peered":"2024-03-07T07:19:16.301033+0000","last_unstale":"2024-03-07T07:19:17.297262+0000","last_undegraded":"2024-03-07T07:19:17.297262+0000","last_fullsized":"2024-03-07T07:19:17.297262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:34.845258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009648+0000","last_change":"2024-03-07T07:19:16.296414+0000","last_active":"2024-03-07T07:19:18.009648+0000","last_peered":"2024-03-07T07:19:18.009648+0000","last_clean":"2024-03-07T07:19:18.009648+0000","last_became_active":"2024-03-07T07:19:16.296078+0000","last_became_peered":"2024-03-07T07:19:16.296078+0000","last_unstale":"2024-03-07T07:19:18.009648+0000","last_undegraded":"2024-03-07T07:19:18.009648+0000","last_fullsized":"2024-03-07T07:19:18.009648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870192+0000","last_change":"2024-03-07T07:19:16.298815+0000","last_active":"2024-03-07T07:19:17.870192+0000","last_peered":"2024-03-07T07:19:17.870192+0000","last_clean":"2024-03-07T07:19:17.870192+0000","last_became_active":"2024-03-07T07:19:16.298560+0000","last_became_peered":"2024-03-07T07:19:16.298560+0000","last_unstale":"2024-03-07T07:19:17.870192+0000","last_undegraded":"2024-03-07T07:19:17.870192+0000","last_fullsized":"2024-03-07T07:19:17.870192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:14:50.701779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293979+0000","last_change":"2024-03-07T07:19:16.304915+0000","last_active":"2024-03-07T07:19:17.293979+0000","last_peered":"2024-03-07T07:19:17.293979+0000","last_clean":"2024-03-07T07:19:17.293979+0000","last_became_active":"2024-03-07T07:19:16.304581+0000","last_became_peered":"2024-03-07T07:19:16.304581+0000","last_unstale":"2024-03-07T07:19:17.293979+0000","last_undegraded":"2024-03-07T07:19:17.293979+0000","last_fullsized":"2024-03-07T07:19:17.293979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417242+0000","last_change":"2024-03-07T07:19:16.312660+0000","last_active":"2024-03-07T07:19:17.417242+0000","last_peered":"2024-03-07T07:19:17.417242+0000","last_clean":"2024-03-07T07:19:17.417242+0000","last_became_active":"2024-03-07T07:19:16.312356+0000","last_became_peered":"2024-03-07T07:19:16.312356+0000","last_unstale":"2024-03-07T07:19:17.417242+0000","last_undegraded":"2024-03-07T07:19:17.417242+0000","last_fullsized":"2024-03-07T07:19:17.417242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417697+0000","last_change":"2024-03-07T07:19:16.311126+0000","last_active":"2024-03-07T07:19:17.417697+0000","last_peered":"2024-03-07T07:19:17.417697+0000","last_clean":"2024-03-07T07:19:17.417697+0000","last_became_active":"2024-03-07T07:19:16.310806+0000","last_became_peered":"2024-03-07T07:19:16.310806+0000","last_unstale":"2024-03-07T07:19:17.417697+0000","last_undegraded":"2024-03-07T07:19:17.417697+0000","last_fullsized":"2024-03-07T07:19:17.417697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068899+0000","last_change":"2024-03-07T07:19:16.296921+0000","last_active":"2024-03-07T07:19:18.068899+0000","last_peered":"2024-03-07T07:19:18.068899+0000","last_clean":"2024-03-07T07:19:18.068899+0000","last_became_active":"2024-03-07T07:19:16.296657+0000","last_became_peered":"2024-03-07T07:19:16.296657+0000","last_unstale":"2024-03-07T07:19:18.068899+0000","last_undegraded":"2024-03-07T07:19:18.068899+0000","last_fullsized":"2024-03-07T07:19:18.068899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069320+0000","last_change":"2024-03-07T07:19:16.301924+0000","last_active":"2024-03-07T07:19:18.069320+0000","last_peered":"2024-03-07T07:19:18.069320+0000","last_clean":"2024-03-07T07:19:18.069320+0000","last_became_active":"2024-03-07T07:19:16.301657+0000","last_became_peered":"2024-03-07T07:19:16.301657+0000","last_unstale":"2024-03-07T07:19:18.069320+0000","last_undegraded":"2024-03-07T07:19:18.069320+0000","last_fullsized":"2024-03-07T07:19:18.069320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:16:48.547021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296490+0000","last_change":"2024-03-07T07:19:16.300281+0000","last_active":"2024-03-07T07:19:17.296490+0000","last_peered":"2024-03-07T07:19:17.296490+0000","last_clean":"2024-03-07T07:19:17.296490+0000","last_became_active":"2024-03-07T07:19:16.299775+0000","last_became_peered":"2024-03-07T07:19:16.299775+0000","last_unstale":"2024-03-07T07:19:17.296490+0000","last_undegraded":"2024-03-07T07:19:17.296490+0000","last_fullsized":"2024-03-07T07:19:17.296490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870510+0000","last_change":"2024-03-07T07:19:16.296327+0000","last_active":"2024-03-07T07:19:17.870510+0000","last_peered":"2024-03-07T07:19:17.870510+0000","last_clean":"2024-03-07T07:19:17.870510+0000","last_became_active":"2024-03-07T07:19:16.295959+0000","last_became_peered":"2024-03-07T07:19:16.295959+0000","last_unstale":"2024-03-07T07:19:17.870510+0000","last_undegraded":"2024-03-07T07:19:17.870510+0000","last_fullsized":"2024-03-07T07:19:17.870510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417371+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:17.417371+0000","last_peered":"2024-03-07T07:19:17.417371+0000","last_clean":"2024-03-07T07:19:17.417371+0000","last_became_active":"2024-03-07T07:19:16.307389+0000","last_became_peered":"2024-03-07T07:19:16.307389+0000","last_unstale":"2024-03-07T07:19:17.417371+0000","last_undegraded":"2024-03-07T07:19:17.417371+0000","last_fullsized":"2024-03-07T07:19:17.417371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009458+0000","last_change":"2024-03-07T07:19:16.295700+0000","last_active":"2024-03-07T07:19:18.009458+0000","last_peered":"2024-03-07T07:19:18.009458+0000","last_clean":"2024-03-07T07:19:18.009458+0000","last_became_active":"2024-03-07T07:19:16.295428+0000","last_became_peered":"2024-03-07T07:19:16.295428+0000","last_unstale":"2024-03-07T07:19:18.009458+0000","last_undegraded":"2024-03-07T07:19:18.009458+0000","last_fullsized":"2024-03-07T07:19:18.009458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417302+0000","last_change":"2024-03-07T07:19:16.305382+0000","last_active":"2024-03-07T07:19:17.417302+0000","last_peered":"2024-03-07T07:19:17.417302+0000","last_clean":"2024-03-07T07:19:17.417302+0000","last_became_active":"2024-03-07T07:19:16.304697+0000","last_became_peered":"2024-03-07T07:19:16.304697+0000","last_unstale":"2024-03-07T07:19:17.417302+0000","last_undegraded":"2024-03-07T07:19:17.417302+0000","last_fullsized":"2024-03-07T07:19:17.417302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009833+0000","last_change":"2024-03-07T07:19:16.295728+0000","last_active":"2024-03-07T07:19:18.009833+0000","last_peered":"2024-03-07T07:19:18.009833+0000","last_clean":"2024-03-07T07:19:18.009833+0000","last_became_active":"2024-03-07T07:19:16.295472+0000","last_became_peered":"2024-03-07T07:19:16.295472+0000","last_unstale":"2024-03-07T07:19:18.009833+0000","last_undegraded":"2024-03-07T07:19:18.009833+0000","last_fullsized":"2024-03-07T07:19:18.009833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296848+0000","last_change":"2024-03-07T07:19:16.297196+0000","last_active":"2024-03-07T07:19:17.296848+0000","last_peered":"2024-03-07T07:19:17.296848+0000","last_clean":"2024-03-07T07:19:17.296848+0000","last_became_active":"2024-03-07T07:19:16.296727+0000","last_became_peered":"2024-03-07T07:19:16.296727+0000","last_unstale":"2024-03-07T07:19:17.296848+0000","last_undegraded":"2024-03-07T07:19:17.296848+0000","last_fullsized":"2024-03-07T07:19:17.296848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294209+0000","last_change":"2024-03-07T07:19:16.299086+0000","last_active":"2024-03-07T07:19:17.294209+0000","last_peered":"2024-03-07T07:19:17.294209+0000","last_clean":"2024-03-07T07:19:17.294209+0000","last_became_active":"2024-03-07T07:19:16.298819+0000","last_became_peered":"2024-03-07T07:19:16.298819+0000","last_unstale":"2024-03-07T07:19:17.294209+0000","last_undegraded":"2024-03-07T07:19:17.294209+0000","last_fullsized":"2024-03-07T07:19:17.294209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009756+0000","last_change":"2024-03-07T07:19:16.299038+0000","last_active":"2024-03-07T07:19:18.009756+0000","last_peered":"2024-03-07T07:19:18.009756+0000","last_clean":"2024-03-07T07:19:18.009756+0000","last_became_active":"2024-03-07T07:19:16.298748+0000","last_became_peered":"2024-03-07T07:19:16.298748+0000","last_unstale":"2024-03-07T07:19:18.009756+0000","last_undegraded":"2024-03-07T07:19:18.009756+0000","last_fullsized":"2024-03-07T07:19:18.009756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870540+0000","last_change":"2024-03-07T07:19:16.293023+0000","last_active":"2024-03-07T07:19:17.870540+0000","last_peered":"2024-03-07T07:19:17.870540+0000","last_clean":"2024-03-07T07:19:17.870540+0000","last_became_active":"2024-03-07T07:19:16.292670+0000","last_became_peered":"2024-03-07T07:19:16.292670+0000","last_unstale":"2024-03-07T07:19:17.870540+0000","last_undegraded":"2024-03-07T07:19:17.870540+0000","last_fullsized":"2024-03-07T07:19:17.870540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417541+0000","last_change":"2024-03-07T07:19:16.310502+0000","last_active":"2024-03-07T07:19:17.417541+0000","last_peered":"2024-03-07T07:19:17.417541+0000","last_clean":"2024-03-07T07:19:17.417541+0000","last_became_active":"2024-03-07T07:19:16.309883+0000","last_became_peered":"2024-03-07T07:19:16.309883+0000","last_unstale":"2024-03-07T07:19:17.417541+0000","last_undegraded":"2024-03-07T07:19:17.417541+0000","last_fullsized":"2024-03-07T07:19:17.417541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068772+0000","last_change":"2024-03-07T07:19:16.306042+0000","last_active":"2024-03-07T07:19:18.068772+0000","last_peered":"2024-03-07T07:19:18.068772+0000","last_clean":"2024-03-07T07:19:18.068772+0000","last_became_active":"2024-03-07T07:19:16.304954+0000","last_became_peered":"2024-03-07T07:19:16.304954+0000","last_unstale":"2024-03-07T07:19:18.068772+0000","last_undegraded":"2024-03-07T07:19:18.068772+0000","last_fullsized":"2024-03-07T07:19:18.068772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296496+0000","last_change":"2024-03-07T07:19:16.300717+0000","last_active":"2024-03-07T07:19:17.296496+0000","last_peered":"2024-03-07T07:19:17.296496+0000","last_clean":"2024-03-07T07:19:17.296496+0000","last_became_active":"2024-03-07T07:19:16.300170+0000","last_became_peered":"2024-03-07T07:19:16.300170+0000","last_unstale":"2024-03-07T07:19:17.296496+0000","last_undegraded":"2024-03-07T07:19:17.296496+0000","last_fullsized":"2024-03-07T07:19:17.296496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297170+0000","last_change":"2024-03-07T07:19:16.299268+0000","last_active":"2024-03-07T07:19:17.297170+0000","last_peered":"2024-03-07T07:19:17.297170+0000","last_clean":"2024-03-07T07:19:17.297170+0000","last_became_active":"2024-03-07T07:19:16.298987+0000","last_became_peered":"2024-03-07T07:19:16.298987+0000","last_unstale":"2024-03-07T07:19:17.297170+0000","last_undegraded":"2024-03-07T07:19:17.297170+0000","last_fullsized":"2024-03-07T07:19:17.297170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:19.940197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009780+0000","last_change":"2024-03-07T07:19:15.298629+0000","last_active":"2024-03-07T07:19:18.009780+0000","last_peered":"2024-03-07T07:19:18.009780+0000","last_clean":"2024-03-07T07:19:18.009780+0000","last_became_active":"2024-03-07T07:19:15.298338+0000","last_became_peered":"2024-03-07T07:19:15.298338+0000","last_unstale":"2024-03-07T07:19:18.009780+0000","last_undegraded":"2024-03-07T07:19:18.009780+0000","last_fullsized":"2024-03-07T07:19:18.009780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870658+0000","last_change":"2024-03-07T07:19:16.299764+0000","last_active":"2024-03-07T07:19:17.870658+0000","last_peered":"2024-03-07T07:19:17.870658+0000","last_clean":"2024-03-07T07:19:17.870658+0000","last_became_active":"2024-03-07T07:19:16.299493+0000","last_became_peered":"2024-03-07T07:19:16.299493+0000","last_unstale":"2024-03-07T07:19:17.870658+0000","last_undegraded":"2024-03-07T07:19:17.870658+0000","last_fullsized":"2024-03-07T07:19:17.870658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296626+0000","last_change":"2024-03-07T07:19:15.285743+0000","last_active":"2024-03-07T07:19:17.296626+0000","last_peered":"2024-03-07T07:19:17.296626+0000","last_clean":"2024-03-07T07:19:17.296626+0000","last_became_active":"2024-03-07T07:19:15.285484+0000","last_became_peered":"2024-03-07T07:19:15.285484+0000","last_unstale":"2024-03-07T07:19:17.296626+0000","last_undegraded":"2024-03-07T07:19:17.296626+0000","last_fullsized":"2024-03-07T07:19:17.296626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009544+0000","last_change":"2024-03-07T07:19:15.294301+0000","last_active":"2024-03-07T07:19:18.009544+0000","last_peered":"2024-03-07T07:19:18.009544+0000","last_clean":"2024-03-07T07:19:18.009544+0000","last_became_active":"2024-03-07T07:19:15.294023+0000","last_became_peered":"2024-03-07T07:19:15.294023+0000","last_unstale":"2024-03-07T07:19:18.009544+0000","last_undegraded":"2024-03-07T07:19:18.009544+0000","last_fullsized":"2024-03-07T07:19:18.009544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009524+0000","last_change":"2024-03-07T07:19:16.299031+0000","last_active":"2024-03-07T07:19:18.009524+0000","last_peered":"2024-03-07T07:19:18.009524+0000","last_clean":"2024-03-07T07:19:18.009524+0000","last_became_active":"2024-03-07T07:19:16.298760+0000","last_became_peered":"2024-03-07T07:19:16.298760+0000","last_unstale":"2024-03-07T07:19:18.009524+0000","last_undegraded":"2024-03-07T07:19:18.009524+0000","last_fullsized":"2024-03-07T07:19:18.009524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417339+0000","last_change":"2024-03-07T07:19:15.292620+0000","last_active":"2024-03-07T07:19:17.417339+0000","last_peered":"2024-03-07T07:19:17.417339+0000","last_clean":"2024-03-07T07:19:17.417339+0000","last_became_active":"2024-03-07T07:19:15.292337+0000","last_became_peered":"2024-03-07T07:19:15.292337+0000","last_unstale":"2024-03-07T07:19:17.417339+0000","last_undegraded":"2024-03-07T07:19:17.417339+0000","last_fullsized":"2024-03-07T07:19:17.417339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293989+0000","last_change":"2024-03-07T07:19:16.297970+0000","last_active":"2024-03-07T07:19:17.293989+0000","last_peered":"2024-03-07T07:19:17.293989+0000","last_clean":"2024-03-07T07:19:17.293989+0000","last_became_active":"2024-03-07T07:19:16.297661+0000","last_became_peered":"2024-03-07T07:19:16.297661+0000","last_unstale":"2024-03-07T07:19:17.293989+0000","last_undegraded":"2024-03-07T07:19:17.293989+0000","last_fullsized":"2024-03-07T07:19:17.293989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296514+0000","last_change":"2024-03-07T07:19:15.295817+0000","last_active":"2024-03-07T07:19:17.296514+0000","last_peered":"2024-03-07T07:19:17.296514+0000","last_clean":"2024-03-07T07:19:17.296514+0000","last_became_active":"2024-03-07T07:19:15.295538+0000","last_became_peered":"2024-03-07T07:19:15.295538+0000","last_unstale":"2024-03-07T07:19:17.296514+0000","last_undegraded":"2024-03-07T07:19:17.296514+0000","last_fullsized":"2024-03-07T07:19:17.296514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417710+0000","last_change":"2024-03-07T07:19:16.307152+0000","last_active":"2024-03-07T07:19:17.417710+0000","last_peered":"2024-03-07T07:19:17.417710+0000","last_clean":"2024-03-07T07:19:17.417710+0000","last_became_active":"2024-03-07T07:19:16.306890+0000","last_became_peered":"2024-03-07T07:19:16.306890+0000","last_unstale":"2024-03-07T07:19:17.417710+0000","last_undegraded":"2024-03-07T07:19:17.417710+0000","last_fullsized":"2024-03-07T07:19:17.417710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417582+0000","last_change":"2024-03-07T07:19:15.301303+0000","last_active":"2024-03-07T07:19:17.417582+0000","last_peered":"2024-03-07T07:19:17.417582+0000","last_clean":"2024-03-07T07:19:17.417582+0000","last_became_active":"2024-03-07T07:19:15.301031+0000","last_became_peered":"2024-03-07T07:19:15.301031+0000","last_unstale":"2024-03-07T07:19:17.417582+0000","last_undegraded":"2024-03-07T07:19:17.417582+0000","last_fullsized":"2024-03-07T07:19:17.417582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870348+0000","last_change":"2024-03-07T07:19:16.305499+0000","last_active":"2024-03-07T07:19:17.870348+0000","last_peered":"2024-03-07T07:19:17.870348+0000","last_clean":"2024-03-07T07:19:17.870348+0000","last_became_active":"2024-03-07T07:19:16.304971+0000","last_became_peered":"2024-03-07T07:19:16.304971+0000","last_unstale":"2024-03-07T07:19:17.870348+0000","last_undegraded":"2024-03-07T07:19:17.870348+0000","last_fullsized":"2024-03-07T07:19:17.870348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294183+0000","last_change":"2024-03-07T07:19:15.293364+0000","last_active":"2024-03-07T07:19:17.294183+0000","last_peered":"2024-03-07T07:19:17.294183+0000","last_clean":"2024-03-07T07:19:17.294183+0000","last_became_active":"2024-03-07T07:19:15.293069+0000","last_became_peered":"2024-03-07T07:19:15.293069+0000","last_unstale":"2024-03-07T07:19:17.294183+0000","last_undegraded":"2024-03-07T07:19:17.294183+0000","last_fullsized":"2024-03-07T07:19:17.294183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297511+0000","last_change":"2024-03-07T07:19:16.304722+0000","last_active":"2024-03-07T07:19:17.297511+0000","last_peered":"2024-03-07T07:19:17.297511+0000","last_clean":"2024-03-07T07:19:17.297511+0000","last_became_active":"2024-03-07T07:19:16.304438+0000","last_became_peered":"2024-03-07T07:19:16.304438+0000","last_unstale":"2024-03-07T07:19:17.297511+0000","last_undegraded":"2024-03-07T07:19:17.297511+0000","last_fullsized":"2024-03-07T07:19:17.297511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296596+0000","last_change":"2024-03-07T07:19:15.292852+0000","last_active":"2024-03-07T07:19:17.296596+0000","last_peered":"2024-03-07T07:19:17.296596+0000","last_clean":"2024-03-07T07:19:17.296596+0000","last_became_active":"2024-03-07T07:19:15.292584+0000","last_became_peered":"2024-03-07T07:19:15.292584+0000","last_unstale":"2024-03-07T07:19:17.296596+0000","last_undegraded":"2024-03-07T07:19:17.296596+0000","last_fullsized":"2024-03-07T07:19:17.296596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068903+0000","last_change":"2024-03-07T07:19:16.303347+0000","last_active":"2024-03-07T07:19:18.068903+0000","last_peered":"2024-03-07T07:19:18.068903+0000","last_clean":"2024-03-07T07:19:18.068903+0000","last_became_active":"2024-03-07T07:19:16.303083+0000","last_became_peered":"2024-03-07T07:19:16.303083+0000","last_unstale":"2024-03-07T07:19:18.068903+0000","last_undegraded":"2024-03-07T07:19:18.068903+0000","last_fullsized":"2024-03-07T07:19:18.068903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870639+0000","last_change":"2024-03-07T07:19:15.296780+0000","last_active":"2024-03-07T07:19:17.870639+0000","last_peered":"2024-03-07T07:19:17.870639+0000","last_clean":"2024-03-07T07:19:17.870639+0000","last_became_active":"2024-03-07T07:19:15.296300+0000","last_became_peered":"2024-03-07T07:19:15.296300+0000","last_unstale":"2024-03-07T07:19:17.870639+0000","last_undegraded":"2024-03-07T07:19:17.870639+0000","last_fullsized":"2024-03-07T07:19:17.870639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870345+0000","last_change":"2024-03-07T07:19:16.297140+0000","last_active":"2024-03-07T07:19:17.870345+0000","last_peered":"2024-03-07T07:19:17.870345+0000","last_clean":"2024-03-07T07:19:17.870345+0000","last_became_active":"2024-03-07T07:19:16.296854+0000","last_became_peered":"2024-03-07T07:19:16.296854+0000","last_unstale":"2024-03-07T07:19:17.870345+0000","last_undegraded":"2024-03-07T07:19:17.870345+0000","last_fullsized":"2024-03-07T07:19:17.870345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417473+0000","last_change":"2024-03-07T07:19:15.299975+0000","last_active":"2024-03-07T07:19:17.417473+0000","last_peered":"2024-03-07T07:19:17.417473+0000","last_clean":"2024-03-07T07:19:17.417473+0000","last_became_active":"2024-03-07T07:19:15.299753+0000","last_became_peered":"2024-03-07T07:19:15.299753+0000","last_unstale":"2024-03-07T07:19:17.417473+0000","last_undegraded":"2024-03-07T07:19:17.417473+0000","last_fullsized":"2024-03-07T07:19:17.417473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068921+0000","last_change":"2024-03-07T07:19:16.307214+0000","last_active":"2024-03-07T07:19:18.068921+0000","last_peered":"2024-03-07T07:19:18.068921+0000","last_clean":"2024-03-07T07:19:18.068921+0000","last_became_active":"2024-03-07T07:19:16.306690+0000","last_became_peered":"2024-03-07T07:19:16.306690+0000","last_unstale":"2024-03-07T07:19:18.068921+0000","last_undegraded":"2024-03-07T07:19:18.068921+0000","last_fullsized":"2024-03-07T07:19:18.068921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296268+0000","last_change":"2024-03-07T07:19:15.286950+0000","last_active":"2024-03-07T07:19:17.296268+0000","last_peered":"2024-03-07T07:19:17.296268+0000","last_clean":"2024-03-07T07:19:17.296268+0000","last_became_active":"2024-03-07T07:19:15.286655+0000","last_became_peered":"2024-03-07T07:19:15.286655+0000","last_unstale":"2024-03-07T07:19:17.296268+0000","last_undegraded":"2024-03-07T07:19:17.296268+0000","last_fullsized":"2024-03-07T07:19:17.296268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296267+0000","last_change":"2024-03-07T07:19:16.299235+0000","last_active":"2024-03-07T07:19:17.296267+0000","last_peered":"2024-03-07T07:19:17.296267+0000","last_clean":"2024-03-07T07:19:17.296267+0000","last_became_active":"2024-03-07T07:19:16.298945+0000","last_became_peered":"2024-03-07T07:19:16.298945+0000","last_unstale":"2024-03-07T07:19:17.296267+0000","last_undegraded":"2024-03-07T07:19:17.296267+0000","last_fullsized":"2024-03-07T07:19:17.296267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068707+0000","last_change":"2024-03-07T07:19:15.289721+0000","last_active":"2024-03-07T07:19:18.068707+0000","last_peered":"2024-03-07T07:19:18.068707+0000","last_clean":"2024-03-07T07:19:18.068707+0000","last_became_active":"2024-03-07T07:19:15.289410+0000","last_became_peered":"2024-03-07T07:19:15.289410+0000","last_unstale":"2024-03-07T07:19:18.068707+0000","last_undegraded":"2024-03-07T07:19:18.068707+0000","last_fullsized":"2024-03-07T07:19:18.068707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068748+0000","last_change":"2024-03-07T07:19:16.306954+0000","last_active":"2024-03-07T07:19:18.068748+0000","last_peered":"2024-03-07T07:19:18.068748+0000","last_clean":"2024-03-07T07:19:18.068748+0000","last_became_active":"2024-03-07T07:19:16.306639+0000","last_became_peered":"2024-03-07T07:19:16.306639+0000","last_unstale":"2024-03-07T07:19:18.068748+0000","last_undegraded":"2024-03-07T07:19:18.068748+0000","last_fullsized":"2024-03-07T07:19:18.068748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297392+0000","last_change":"2024-03-07T07:19:15.297356+0000","last_active":"2024-03-07T07:19:17.297392+0000","last_peered":"2024-03-07T07:19:17.297392+0000","last_clean":"2024-03-07T07:19:17.297392+0000","last_became_active":"2024-03-07T07:19:15.296989+0000","last_became_peered":"2024-03-07T07:19:15.296989+0000","last_unstale":"2024-03-07T07:19:17.297392+0000","last_undegraded":"2024-03-07T07:19:17.297392+0000","last_fullsized":"2024-03-07T07:19:17.297392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:17:57.790720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069126+0000","last_change":"2024-03-07T07:19:16.306845+0000","last_active":"2024-03-07T07:19:18.069126+0000","last_peered":"2024-03-07T07:19:18.069126+0000","last_clean":"2024-03-07T07:19:18.069126+0000","last_became_active":"2024-03-07T07:19:16.306406+0000","last_became_peered":"2024-03-07T07:19:16.306406+0000","last_unstale":"2024-03-07T07:19:18.069126+0000","last_undegraded":"2024-03-07T07:19:18.069126+0000","last_fullsized":"2024-03-07T07:19:18.069126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068846+0000","last_change":"2024-03-07T07:19:15.293365+0000","last_active":"2024-03-07T07:19:18.068846+0000","last_peered":"2024-03-07T07:19:18.068846+0000","last_clean":"2024-03-07T07:19:18.068846+0000","last_became_active":"2024-03-07T07:19:15.293062+0000","last_became_peered":"2024-03-07T07:19:15.293062+0000","last_unstale":"2024-03-07T07:19:18.068846+0000","last_undegraded":"2024-03-07T07:19:18.068846+0000","last_fullsized":"2024-03-07T07:19:18.068846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069160+0000","last_change":"2024-03-07T07:19:16.307514+0000","last_active":"2024-03-07T07:19:18.069160+0000","last_peered":"2024-03-07T07:19:18.069160+0000","last_clean":"2024-03-07T07:19:18.069160+0000","last_became_active":"2024-03-07T07:19:16.307226+0000","last_became_peered":"2024-03-07T07:19:16.307226+0000","last_unstale":"2024-03-07T07:19:18.069160+0000","last_undegraded":"2024-03-07T07:19:18.069160+0000","last_fullsized":"2024-03-07T07:19:18.069160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:25:12.269915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294284+0000","last_change":"2024-03-07T07:19:15.290118+0000","last_active":"2024-03-07T07:19:17.294284+0000","last_peered":"2024-03-07T07:19:17.294284+0000","last_clean":"2024-03-07T07:19:17.294284+0000","last_became_active":"2024-03-07T07:19:15.289812+0000","last_became_peered":"2024-03-07T07:19:15.289812+0000","last_unstale":"2024-03-07T07:19:17.294284+0000","last_undegraded":"2024-03-07T07:19:17.294284+0000","last_fullsized":"2024-03-07T07:19:17.294284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417670+0000","last_change":"2024-03-07T07:19:16.303739+0000","last_active":"2024-03-07T07:19:17.417670+0000","last_peered":"2024-03-07T07:19:17.417670+0000","last_clean":"2024-03-07T07:19:17.417670+0000","last_became_active":"2024-03-07T07:19:16.303279+0000","last_became_peered":"2024-03-07T07:19:16.303279+0000","last_unstale":"2024-03-07T07:19:17.417670+0000","last_undegraded":"2024-03-07T07:19:17.417670+0000","last_fullsized":"2024-03-07T07:19:17.417670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068866+0000","last_change":"2024-03-07T07:19:15.296198+0000","last_active":"2024-03-07T07:19:18.068866+0000","last_peered":"2024-03-07T07:19:18.068866+0000","last_clean":"2024-03-07T07:19:18.068866+0000","last_became_active":"2024-03-07T07:19:15.295886+0000","last_became_peered":"2024-03-07T07:19:15.295886+0000","last_unstale":"2024-03-07T07:19:18.068866+0000","last_undegraded":"2024-03-07T07:19:18.068866+0000","last_fullsized":"2024-03-07T07:19:18.068866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069075+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:18.069075+0000","last_peered":"2024-03-07T07:19:18.069075+0000","last_clean":"2024-03-07T07:19:18.069075+0000","last_became_active":"2024-03-07T07:19:16.307119+0000","last_became_peered":"2024-03-07T07:19:16.307119+0000","last_unstale":"2024-03-07T07:19:18.069075+0000","last_undegraded":"2024-03-07T07:19:18.069075+0000","last_fullsized":"2024-03-07T07:19:18.069075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068695+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:19:18.068695+0000","last_peered":"2024-03-07T07:19:18.068695+0000","last_clean":"2024-03-07T07:19:18.068695+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:19:18.068695+0000","last_undegraded":"2024-03-07T07:19:18.068695+0000","last_fullsized":"2024-03-07T07:19:18.068695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417348+0000","last_change":"2024-03-07T07:19:15.295100+0000","last_active":"2024-03-07T07:19:17.417348+0000","last_peered":"2024-03-07T07:19:17.417348+0000","last_clean":"2024-03-07T07:19:17.417348+0000","last_became_active":"2024-03-07T07:19:15.294643+0000","last_became_peered":"2024-03-07T07:19:15.294643+0000","last_unstale":"2024-03-07T07:19:17.417348+0000","last_undegraded":"2024-03-07T07:19:17.417348+0000","last_fullsized":"2024-03-07T07:19:17.417348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.306837+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.305695+0000","last_became_peered":"2024-03-07T07:19:16.305695+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068642+0000","last_change":"2024-03-07T07:19:15.301442+0000","last_active":"2024-03-07T07:19:18.068642+0000","last_peered":"2024-03-07T07:19:18.068642+0000","last_clean":"2024-03-07T07:19:18.068642+0000","last_became_active":"2024-03-07T07:19:15.300526+0000","last_became_peered":"2024-03-07T07:19:15.300526+0000","last_unstale":"2024-03-07T07:19:18.068642+0000","last_undegraded":"2024-03-07T07:19:18.068642+0000","last_fullsized":"2024-03-07T07:19:18.068642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870083+0000","last_change":"2024-03-07T07:19:16.297399+0000","last_active":"2024-03-07T07:19:17.870083+0000","last_peered":"2024-03-07T07:19:17.870083+0000","last_clean":"2024-03-07T07:19:17.870083+0000","last_became_active":"2024-03-07T07:19:16.296941+0000","last_became_peered":"2024-03-07T07:19:16.296941+0000","last_unstale":"2024-03-07T07:19:17.870083+0000","last_undegraded":"2024-03-07T07:19:17.870083+0000","last_fullsized":"2024-03-07T07:19:17.870083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:10.125663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068824+0000","last_change":"2024-03-07T07:19:15.291261+0000","last_active":"2024-03-07T07:19:18.068824+0000","last_peered":"2024-03-07T07:19:18.068824+0000","last_clean":"2024-03-07T07:19:18.068824+0000","last_became_active":"2024-03-07T07:19:15.290977+0000","last_became_peered":"2024-03-07T07:19:15.290977+0000","last_unstale":"2024-03-07T07:19:18.068824+0000","last_undegraded":"2024-03-07T07:19:18.068824+0000","last_fullsized":"2024-03-07T07:19:18.068824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294009+0000","last_change":"2024-03-07T07:19:16.298068+0000","last_active":"2024-03-07T07:19:17.294009+0000","last_peered":"2024-03-07T07:19:17.294009+0000","last_clean":"2024-03-07T07:19:17.294009+0000","last_became_active":"2024-03-07T07:19:16.297795+0000","last_became_peered":"2024-03-07T07:19:16.297795+0000","last_unstale":"2024-03-07T07:19:17.294009+0000","last_undegraded":"2024-03-07T07:19:17.294009+0000","last_fullsized":"2024-03-07T07:19:17.294009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417588+0000","last_change":"2024-03-07T07:19:15.296795+0000","last_active":"2024-03-07T07:19:17.417588+0000","last_peered":"2024-03-07T07:19:17.417588+0000","last_clean":"2024-03-07T07:19:17.417588+0000","last_became_active":"2024-03-07T07:19:15.296534+0000","last_became_peered":"2024-03-07T07:19:15.296534+0000","last_unstale":"2024-03-07T07:19:17.417588+0000","last_undegraded":"2024-03-07T07:19:17.417588+0000","last_fullsized":"2024-03-07T07:19:17.417588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297568+0000","last_change":"2024-03-07T07:19:16.295673+0000","last_active":"2024-03-07T07:19:17.297568+0000","last_peered":"2024-03-07T07:19:17.297568+0000","last_clean":"2024-03-07T07:19:17.297568+0000","last_became_active":"2024-03-07T07:19:16.295139+0000","last_became_peered":"2024-03-07T07:19:16.295139+0000","last_unstale":"2024-03-07T07:19:17.297568+0000","last_undegraded":"2024-03-07T07:19:17.297568+0000","last_fullsized":"2024-03-07T07:19:17.297568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294148+0000","last_change":"2024-03-07T07:19:15.297274+0000","last_active":"2024-03-07T07:19:17.294148+0000","last_peered":"2024-03-07T07:19:17.294148+0000","last_clean":"2024-03-07T07:19:17.294148+0000","last_became_active":"2024-03-07T07:19:15.297000+0000","last_became_peered":"2024-03-07T07:19:15.297000+0000","last_unstale":"2024-03-07T07:19:17.294148+0000","last_undegraded":"2024-03-07T07:19:17.294148+0000","last_fullsized":"2024-03-07T07:19:17.294148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417617+0000","last_change":"2024-03-07T07:19:16.305048+0000","last_active":"2024-03-07T07:19:17.417617+0000","last_peered":"2024-03-07T07:19:17.417617+0000","last_clean":"2024-03-07T07:19:17.417617+0000","last_became_active":"2024-03-07T07:19:16.303993+0000","last_became_peered":"2024-03-07T07:19:16.303993+0000","last_unstale":"2024-03-07T07:19:17.417617+0000","last_undegraded":"2024-03-07T07:19:17.417617+0000","last_fullsized":"2024-03-07T07:19:17.417617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069397+0000","last_change":"2024-03-07T07:19:15.300909+0000","last_active":"2024-03-07T07:19:18.069397+0000","last_peered":"2024-03-07T07:19:18.069397+0000","last_clean":"2024-03-07T07:19:18.069397+0000","last_became_active":"2024-03-07T07:19:15.300573+0000","last_became_peered":"2024-03-07T07:19:15.300573+0000","last_unstale":"2024-03-07T07:19:18.069397+0000","last_undegraded":"2024-03-07T07:19:18.069397+0000","last_fullsized":"2024-03-07T07:19:18.069397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294190+0000","last_change":"2024-03-07T07:19:16.298525+0000","last_active":"2024-03-07T07:19:17.294190+0000","last_peered":"2024-03-07T07:19:17.294190+0000","last_clean":"2024-03-07T07:19:17.294190+0000","last_became_active":"2024-03-07T07:19:16.298264+0000","last_became_peered":"2024-03-07T07:19:16.298264+0000","last_unstale":"2024-03-07T07:19:17.294190+0000","last_undegraded":"2024-03-07T07:19:17.294190+0000","last_fullsized":"2024-03-07T07:19:17.294190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.295564+0000","last_change":"2024-03-07T07:19:15.291265+0000","last_active":"2024-03-07T07:19:17.295564+0000","last_peered":"2024-03-07T07:19:17.295564+0000","last_clean":"2024-03-07T07:19:17.295564+0000","last_became_active":"2024-03-07T07:19:15.290889+0000","last_became_peered":"2024-03-07T07:19:15.290889+0000","last_unstale":"2024-03-07T07:19:17.295564+0000","last_undegraded":"2024-03-07T07:19:17.295564+0000","last_fullsized":"2024-03-07T07:19:17.295564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009905+0000","last_change":"2024-03-07T07:19:16.303881+0000","last_active":"2024-03-07T07:19:18.009905+0000","last_peered":"2024-03-07T07:19:18.009905+0000","last_clean":"2024-03-07T07:19:18.009905+0000","last_became_active":"2024-03-07T07:19:16.303597+0000","last_became_peered":"2024-03-07T07:19:16.303597+0000","last_unstale":"2024-03-07T07:19:18.009905+0000","last_undegraded":"2024-03-07T07:19:18.009905+0000","last_fullsized":"2024-03-07T07:19:18.009905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296735+0000","last_change":"2024-03-07T07:19:15.296436+0000","last_active":"2024-03-07T07:19:17.296735+0000","last_peered":"2024-03-07T07:19:17.296735+0000","last_clean":"2024-03-07T07:19:17.296735+0000","last_became_active":"2024-03-07T07:19:15.296152+0000","last_became_peered":"2024-03-07T07:19:15.296152+0000","last_unstale":"2024-03-07T07:19:17.296735+0000","last_undegraded":"2024-03-07T07:19:17.296735+0000","last_fullsized":"2024-03-07T07:19:17.296735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870218+0000","last_change":"2024-03-07T07:19:16.300794+0000","last_active":"2024-03-07T07:19:17.870218+0000","last_peered":"2024-03-07T07:19:17.870218+0000","last_clean":"2024-03-07T07:19:17.870218+0000","last_became_active":"2024-03-07T07:19:16.300531+0000","last_became_peered":"2024-03-07T07:19:16.300531+0000","last_unstale":"2024-03-07T07:19:17.870218+0000","last_undegraded":"2024-03-07T07:19:17.870218+0000","last_fullsized":"2024-03-07T07:19:17.870218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068653+0000","last_change":"2024-03-07T07:19:15.301871+0000","last_active":"2024-03-07T07:19:18.068653+0000","last_peered":"2024-03-07T07:19:18.068653+0000","last_clean":"2024-03-07T07:19:18.068653+0000","last_became_active":"2024-03-07T07:19:15.301269+0000","last_became_peered":"2024-03-07T07:19:15.301269+0000","last_unstale":"2024-03-07T07:19:18.068653+0000","last_undegraded":"2024-03-07T07:19:18.068653+0000","last_fullsized":"2024-03-07T07:19:18.068653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297261+0000","last_change":"2024-03-07T07:19:16.293571+0000","last_active":"2024-03-07T07:19:17.297261+0000","last_peered":"2024-03-07T07:19:17.297261+0000","last_clean":"2024-03-07T07:19:17.297261+0000","last_became_active":"2024-03-07T07:19:16.293217+0000","last_became_peered":"2024-03-07T07:19:16.293217+0000","last_unstale":"2024-03-07T07:19:17.297261+0000","last_undegraded":"2024-03-07T07:19:17.297261+0000","last_fullsized":"2024-03-07T07:19:17.297261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:11.899192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294039+0000","last_change":"2024-03-07T07:19:15.291033+0000","last_active":"2024-03-07T07:19:17.294039+0000","last_peered":"2024-03-07T07:19:17.294039+0000","last_clean":"2024-03-07T07:19:17.294039+0000","last_became_active":"2024-03-07T07:19:15.290699+0000","last_became_peered":"2024-03-07T07:19:15.290699+0000","last_unstale":"2024-03-07T07:19:17.294039+0000","last_undegraded":"2024-03-07T07:19:17.294039+0000","last_fullsized":"2024-03-07T07:19:17.294039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009889+0000","last_change":"2024-03-07T07:19:16.300154+0000","last_active":"2024-03-07T07:19:18.009889+0000","last_peered":"2024-03-07T07:19:18.009889+0000","last_clean":"2024-03-07T07:19:18.009889+0000","last_became_active":"2024-03-07T07:19:16.299831+0000","last_became_peered":"2024-03-07T07:19:16.299831+0000","last_unstale":"2024-03-07T07:19:18.009889+0000","last_undegraded":"2024-03-07T07:19:18.009889+0000","last_fullsized":"2024-03-07T07:19:18.009889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294117+0000","last_change":"2024-03-07T07:19:15.292788+0000","last_active":"2024-03-07T07:19:17.294117+0000","last_peered":"2024-03-07T07:19:17.294117+0000","last_clean":"2024-03-07T07:19:17.294117+0000","last_became_active":"2024-03-07T07:19:15.292478+0000","last_became_peered":"2024-03-07T07:19:15.292478+0000","last_unstale":"2024-03-07T07:19:17.294117+0000","last_undegraded":"2024-03-07T07:19:17.294117+0000","last_fullsized":"2024-03-07T07:19:17.294117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.302180+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069079+0000","last_change":"2024-03-07T07:19:15.300817+0000","last_active":"2024-03-07T07:19:18.069079+0000","last_peered":"2024-03-07T07:19:18.069079+0000","last_clean":"2024-03-07T07:19:18.069079+0000","last_became_active":"2024-03-07T07:19:15.300502+0000","last_became_peered":"2024-03-07T07:19:15.300502+0000","last_unstale":"2024-03-07T07:19:18.069079+0000","last_undegraded":"2024-03-07T07:19:18.069079+0000","last_fullsized":"2024-03-07T07:19:18.069079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296526+0000","last_change":"2024-03-07T07:19:16.300613+0000","last_active":"2024-03-07T07:19:17.296526+0000","last_peered":"2024-03-07T07:19:17.296526+0000","last_clean":"2024-03-07T07:19:17.296526+0000","last_became_active":"2024-03-07T07:19:16.299969+0000","last_became_peered":"2024-03-07T07:19:16.299969+0000","last_unstale":"2024-03-07T07:19:17.296526+0000","last_undegraded":"2024-03-07T07:19:17.296526+0000","last_fullsized":"2024-03-07T07:19:17.296526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417649+0000","last_change":"2024-03-07T07:19:16.316583+0000","last_active":"2024-03-07T07:19:17.417649+0000","last_peered":"2024-03-07T07:19:17.417649+0000","last_clean":"2024-03-07T07:19:17.417649+0000","last_became_active":"2024-03-07T07:19:16.316351+0000","last_became_peered":"2024-03-07T07:19:16.316351+0000","last_unstale":"2024-03-07T07:19:17.417649+0000","last_undegraded":"2024-03-07T07:19:17.417649+0000","last_fullsized":"2024-03-07T07:19:17.417649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069099+0000","last_change":"2024-03-07T07:19:15.300879+0000","last_active":"2024-03-07T07:19:18.069099+0000","last_peered":"2024-03-07T07:19:18.069099+0000","last_clean":"2024-03-07T07:19:18.069099+0000","last_became_active":"2024-03-07T07:19:15.300594+0000","last_became_peered":"2024-03-07T07:19:15.300594+0000","last_unstale":"2024-03-07T07:19:18.069099+0000","last_undegraded":"2024-03-07T07:19:18.069099+0000","last_fullsized":"2024-03-07T07:19:18.069099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417463+0000","last_change":"2024-03-07T07:19:16.313560+0000","last_active":"2024-03-07T07:19:17.417463+0000","last_peered":"2024-03-07T07:19:17.417463+0000","last_clean":"2024-03-07T07:19:17.417463+0000","last_became_active":"2024-03-07T07:19:16.313130+0000","last_became_peered":"2024-03-07T07:19:16.313130+0000","last_unstale":"2024-03-07T07:19:17.417463+0000","last_undegraded":"2024-03-07T07:19:17.417463+0000","last_fullsized":"2024-03-07T07:19:17.417463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069186+0000","last_change":"2024-03-07T07:19:15.301563+0000","last_active":"2024-03-07T07:19:18.069186+0000","last_peered":"2024-03-07T07:19:18.069186+0000","last_clean":"2024-03-07T07:19:18.069186+0000","last_became_active":"2024-03-07T07:19:15.301282+0000","last_became_peered":"2024-03-07T07:19:15.301282+0000","last_unstale":"2024-03-07T07:19:18.069186+0000","last_undegraded":"2024-03-07T07:19:18.069186+0000","last_fullsized":"2024-03-07T07:19:18.069186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870211+0000","last_change":"2024-03-07T07:19:16.306438+0000","last_active":"2024-03-07T07:19:17.870211+0000","last_peered":"2024-03-07T07:19:17.870211+0000","last_clean":"2024-03-07T07:19:17.870211+0000","last_became_active":"2024-03-07T07:19:16.305209+0000","last_became_peered":"2024-03-07T07:19:16.305209+0000","last_unstale":"2024-03-07T07:19:17.870211+0000","last_undegraded":"2024-03-07T07:19:17.870211+0000","last_fullsized":"2024-03-07T07:19:17.870211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009925+0000","last_change":"2024-03-07T07:19:15.299866+0000","last_active":"2024-03-07T07:19:18.009925+0000","last_peered":"2024-03-07T07:19:18.009925+0000","last_clean":"2024-03-07T07:19:18.009925+0000","last_became_active":"2024-03-07T07:19:15.299621+0000","last_became_peered":"2024-03-07T07:19:15.299621+0000","last_unstale":"2024-03-07T07:19:18.009925+0000","last_undegraded":"2024-03-07T07:19:18.009925+0000","last_fullsized":"2024-03-07T07:19:18.009925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009615+0000","last_change":"2024-03-07T07:19:16.296975+0000","last_active":"2024-03-07T07:19:18.009615+0000","last_peered":"2024-03-07T07:19:18.009615+0000","last_clean":"2024-03-07T07:19:18.009615+0000","last_became_active":"2024-03-07T07:19:16.296702+0000","last_became_peered":"2024-03-07T07:19:16.296702+0000","last_unstale":"2024-03-07T07:19:18.009615+0000","last_undegraded":"2024-03-07T07:19:18.009615+0000","last_fullsized":"2024-03-07T07:19:18.009615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870446+0000","last_change":"2024-03-07T07:19:15.303925+0000","last_active":"2024-03-07T07:19:17.870446+0000","last_peered":"2024-03-07T07:19:17.870446+0000","last_clean":"2024-03-07T07:19:17.870446+0000","last_became_active":"2024-03-07T07:19:15.303646+0000","last_became_peered":"2024-03-07T07:19:15.303646+0000","last_unstale":"2024-03-07T07:19:17.870446+0000","last_undegraded":"2024-03-07T07:19:17.870446+0000","last_fullsized":"2024-03-07T07:19:17.870446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:19:26.792 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:19:26.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.930+0000 7f2af3799640 1 Processor -- start 2024-03-07T07:19:26.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.930+0000 7f2af3799640 1 -- start start 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af3799640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec144540 0x7f2aec162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af3799640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2aec162ab0 0x7f2aec071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af3799640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0b4bd0 con 0x7f2aec0b5dd0 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af3799640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aec141700 con 0x7f2aec162ab0 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af3799640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec161e00 con 0x7f2aec144540 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af1535640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f2aec0b5dd0 0x7f2aec0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48728/0 (socket says 172.21.15.64:48728) 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af1535640 1 -- 172.21.15.64:0/800306121 learned_addr learned my addr 172.21.15.64:0/800306121 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af1d36640 1 --2- 172.21.15.64:0/800306121 >> v2:172.21.15.64:3300/0 conn(0x7f2aec162ab0 0x7f2aec071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.931+0000 7f2af0d34640 1 --2- 172.21.15.64:0/800306121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec144540 0x7f2aec162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2aebfff640 1 -- 172.21.15.64:0/800306121 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559169547 0 0) 0x7f2aec0b4bd0 con 0x7f2aec0b5dd0 2024-03-07T07:19:26.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2aebfff640 1 -- 172.21.15.64:0/800306121 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad4003520 con 0x7f2aec0b5dd0 2024-03-07T07:19:26.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2af1d36640 1 -- 172.21.15.64:0/800306121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec144540 msgr2=0x7f2aec162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:26.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2af1d36640 1 --2- 172.21.15.64:0/800306121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec144540 0x7f2aec162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2af1d36640 1 -- 172.21.15.64:0/800306121 >> v1:172.21.15.64:6789/0 conn(0x7f2aec0b5dd0 legacy=0x7f2aec0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2af1d36640 1 -- 172.21.15.64:0/800306121 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0b4eb0 con 0x7f2aec162ab0 2024-03-07T07:19:26.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2af1d36640 1 --2- 172.21.15.64:0/800306121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec162ab0 0x7f2aec071410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009d90 tx=0x7f2adc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802a4211ac085c91 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2aebfff640 1 -- 172.21.15.64:0/800306121 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc039070 con 0x7f2aec162ab0 2024-03-07T07:19:26.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2aebfff640 1 -- 172.21.15.64:0/800306121 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc02cbc0 con 0x7f2aec162ab0 2024-03-07T07:19:26.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.932+0000 7f2aebfff640 1 -- 172.21.15.64:0/800306121 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc034440 con 0x7f2aec162ab0 2024-03-07T07:19:26.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 -- 172.21.15.64:0/800306121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec162ab0 msgr2=0x7f2aec071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 --2- 172.21.15.64:0/800306121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec162ab0 0x7f2aec071410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009d90 tx=0x7f2adc02be10 comp rx=0 tx=0).stop 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 -- 172.21.15.64:0/800306121 shutdown_connections 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 --2- 172.21.15.64:0/800306121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec162ab0 0x7f2aec071410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 --2- 172.21.15.64:0/800306121 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec144540 0x7f2aec162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 -- 172.21.15.64:0/800306121 >> 172.21.15.64:0/800306121 conn(0x7f2aec0d0e70 msgr2=0x7f2aec0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 -- 172.21.15.64:0/800306121 shutdown_connections 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.933+0000 7f2af3799640 1 -- 172.21.15.64:0/800306121 wait complete. 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.934+0000 7f2af3799640 1 Processor -- start 2024-03-07T07:19:26.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.934+0000 7f2af3799640 1 -- start start 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af3799640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec0b5dd0 0x7f2aec13dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af3799640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec144540 0x7f2aec1428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af3799640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2aec15b930 0x7f2aec1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af3799640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0b61b0 con 0x7f2aec144540 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af3799640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec162050 con 0x7f2aec15b930 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af3799640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec071950 con 0x7f2aec0b5dd0 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec144540 0x7f2aec1428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec144540 0x7f2aec1428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34584/0 (socket says 172.21.15.64:34584) 2024-03-07T07:19:26.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 -- 172.21.15.64:0/2412966298 learned_addr learned my addr 172.21.15.64:0/2412966298 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af1535640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec0b5dd0 0x7f2aec13dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af1d36640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2aec15b930 0x7f2aec1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 -- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec0b5dd0 msgr2=0x7f2aec13dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec0b5dd0 0x7f2aec13dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 -- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2aec15b930 msgr2=0x7f2aec1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2aec15b930 0x7f2aec1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.935+0000 7f2af0d34640 1 -- 172.21.15.64:0/2412966298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec1e95d0 con 0x7f2aec144540 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.936+0000 7f2af0d34640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec144540 0x7f2aec1428d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000e9b0 tx=0x7f2ae000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.936+0000 7f2ae9ffb640 1 -- 172.21.15.64:0/2412966298 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0017070 con 0x7f2aec144540 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.936+0000 7f2ae9ffb640 1 -- 172.21.15.64:0/2412966298 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae000c9a0 con 0x7f2aec144540 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.936+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec1e9890 con 0x7f2aec144540 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.936+0000 7f2ae9ffb640 1 -- 172.21.15.64:0/2412966298 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000cd90 con 0x7f2aec144540 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.936+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec1e9e50 con 0x7f2aec144540 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.937+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec062c90 con 0x7f2aec144540 2024-03-07T07:19:26.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.937+0000 7f2ae9ffb640 1 -- 172.21.15.64:0/2412966298 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2ae001a070 con 0x7f2aec144540 2024-03-07T07:19:26.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.938+0000 7f2ae9ffb640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2ad806b4d0 0x7f2ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:26.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.938+0000 7f2ae9ffb640 1 -- 172.21.15.64:0/2412966298 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ae008a140 con 0x7f2aec144540 2024-03-07T07:19:26.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.938+0000 7f2af1535640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2ad806b4d0 0x7f2ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:26.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.938+0000 7f2af1535640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2ad806b4d0 0x7f2ad806d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4005880 tx=0x7f2ae4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:26.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:26.942+0000 7f2ae9ffb640 1 -- 172.21.15.64:0/2412966298 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ae0058970 con 0x7f2aec144540 2024-03-07T07:19:27.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.119+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2aec071ea0 con 0x7f2ad806b4d0 2024-03-07T07:19:27.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.125+0000 7f2ae9ffb640 1 -- 172.21.15.64:0/2412966298 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f2aec071ea0 con 0x7f2ad806b4d0 2024-03-07T07:19:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:27.129 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:19:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2ad806b4d0 msgr2=0x7f2ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2ad806b4d0 0x7f2ad806d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4005880 tx=0x7f2ae4009040 comp rx=0 tx=0).stop 2024-03-07T07:19:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec144540 msgr2=0x7f2aec1428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec144540 0x7f2aec1428d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000e9b0 tx=0x7f2ae000ee70 comp rx=0 tx=0).stop 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 shutdown_connections 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2ad806b4d0 0x7f2ad806d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2aec144540 0x7f2aec1428d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2aec15b930 0x7f2aec1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 --2- 172.21.15.64:0/2412966298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2aec0b5dd0 0x7f2aec13dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 >> 172.21.15.64:0/2412966298 conn(0x7f2aec0d0e70 msgr2=0x7f2aec0b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 shutdown_connections 2024-03-07T07:19:27.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.128+0000 7f2af3799640 1 -- 172.21.15.64:0/2412966298 wait complete. 2024-03-07T07:19:27.138 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-07T07:19:26.181519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.894564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069031+0000","last_change":"2024-03-07T07:19:16.302696+0000","last_active":"2024-03-07T07:19:18.069031+0000","last_peered":"2024-03-07T07:19:18.069031+0000","last_clean":"2024-03-07T07:19:18.069031+0000","last_became_active":"2024-03-07T07:19:16.302389+0000","last_became_peered":"2024-03-07T07:19:16.302389+0000","last_unstale":"2024-03-07T07:19:18.069031+0000","last_undegraded":"2024-03-07T07:19:18.069031+0000","last_fullsized":"2024-03-07T07:19:18.069031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294255+0000","last_change":"2024-03-07T07:19:16.303217+0000","last_active":"2024-03-07T07:19:17.294255+0000","last_peered":"2024-03-07T07:19:17.294255+0000","last_clean":"2024-03-07T07:19:17.294255+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:17.294255+0000","last_undegraded":"2024-03-07T07:19:17.294255+0000","last_fullsized":"2024-03-07T07:19:17.294255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870246+0000","last_change":"2024-03-07T07:19:16.292858+0000","last_active":"2024-03-07T07:19:17.870246+0000","last_peered":"2024-03-07T07:19:17.870246+0000","last_clean":"2024-03-07T07:19:17.870246+0000","last_became_active":"2024-03-07T07:19:16.292593+0000","last_became_peered":"2024-03-07T07:19:16.292593+0000","last_unstale":"2024-03-07T07:19:17.870246+0000","last_undegraded":"2024-03-07T07:19:17.870246+0000","last_fullsized":"2024-03-07T07:19:17.870246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417391+0000","last_change":"2024-03-07T07:19:16.310590+0000","last_active":"2024-03-07T07:19:17.417391+0000","last_peered":"2024-03-07T07:19:17.417391+0000","last_clean":"2024-03-07T07:19:17.417391+0000","last_became_active":"2024-03-07T07:19:16.310298+0000","last_became_peered":"2024-03-07T07:19:16.310298+0000","last_unstale":"2024-03-07T07:19:17.417391+0000","last_undegraded":"2024-03-07T07:19:17.417391+0000","last_fullsized":"2024-03-07T07:19:17.417391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069132+0000","last_change":"2024-03-07T07:19:16.299586+0000","last_active":"2024-03-07T07:19:18.069132+0000","last_peered":"2024-03-07T07:19:18.069132+0000","last_clean":"2024-03-07T07:19:18.069132+0000","last_became_active":"2024-03-07T07:19:16.299265+0000","last_became_peered":"2024-03-07T07:19:16.299265+0000","last_unstale":"2024-03-07T07:19:18.069132+0000","last_undegraded":"2024-03-07T07:19:18.069132+0000","last_fullsized":"2024-03-07T07:19:18.069132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417301+0000","last_change":"2024-03-07T07:19:16.312616+0000","last_active":"2024-03-07T07:19:17.417301+0000","last_peered":"2024-03-07T07:19:17.417301+0000","last_clean":"2024-03-07T07:19:17.417301+0000","last_became_active":"2024-03-07T07:19:16.312339+0000","last_became_peered":"2024-03-07T07:19:16.312339+0000","last_unstale":"2024-03-07T07:19:17.417301+0000","last_undegraded":"2024-03-07T07:19:17.417301+0000","last_fullsized":"2024-03-07T07:19:17.417301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068749+0000","last_change":"2024-03-07T07:19:16.302351+0000","last_active":"2024-03-07T07:19:18.068749+0000","last_peered":"2024-03-07T07:19:18.068749+0000","last_clean":"2024-03-07T07:19:18.068749+0000","last_became_active":"2024-03-07T07:19:16.302071+0000","last_became_peered":"2024-03-07T07:19:16.302071+0000","last_unstale":"2024-03-07T07:19:18.068749+0000","last_undegraded":"2024-03-07T07:19:18.068749+0000","last_fullsized":"2024-03-07T07:19:18.068749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009417+0000","last_change":"2024-03-07T07:19:16.310400+0000","last_active":"2024-03-07T07:19:18.009417+0000","last_peered":"2024-03-07T07:19:18.009417+0000","last_clean":"2024-03-07T07:19:18.009417+0000","last_became_active":"2024-03-07T07:19:16.310066+0000","last_became_peered":"2024-03-07T07:19:16.310066+0000","last_unstale":"2024-03-07T07:19:18.009417+0000","last_undegraded":"2024-03-07T07:19:18.009417+0000","last_fullsized":"2024-03-07T07:19:18.009417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069013+0000","last_change":"2024-03-07T07:19:16.307888+0000","last_active":"2024-03-07T07:19:18.069013+0000","last_peered":"2024-03-07T07:19:18.069013+0000","last_clean":"2024-03-07T07:19:18.069013+0000","last_became_active":"2024-03-07T07:19:16.306871+0000","last_became_peered":"2024-03-07T07:19:16.306871+0000","last_unstale":"2024-03-07T07:19:18.069013+0000","last_undegraded":"2024-03-07T07:19:18.069013+0000","last_fullsized":"2024-03-07T07:19:18.069013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296721+0000","last_change":"2024-03-07T07:19:16.306115+0000","last_active":"2024-03-07T07:19:17.296721+0000","last_peered":"2024-03-07T07:19:17.296721+0000","last_clean":"2024-03-07T07:19:17.296721+0000","last_became_active":"2024-03-07T07:19:16.305591+0000","last_became_peered":"2024-03-07T07:19:16.305591+0000","last_unstale":"2024-03-07T07:19:17.296721+0000","last_undegraded":"2024-03-07T07:19:17.296721+0000","last_fullsized":"2024-03-07T07:19:17.296721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296408+0000","last_change":"2024-03-07T07:19:16.305713+0000","last_active":"2024-03-07T07:19:17.296408+0000","last_peered":"2024-03-07T07:19:17.296408+0000","last_clean":"2024-03-07T07:19:17.296408+0000","last_became_active":"2024-03-07T07:19:16.305408+0000","last_became_peered":"2024-03-07T07:19:16.305408+0000","last_unstale":"2024-03-07T07:19:17.296408+0000","last_undegraded":"2024-03-07T07:19:17.296408+0000","last_fullsized":"2024-03-07T07:19:17.296408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297262+0000","last_change":"2024-03-07T07:19:16.301659+0000","last_active":"2024-03-07T07:19:17.297262+0000","last_peered":"2024-03-07T07:19:17.297262+0000","last_clean":"2024-03-07T07:19:17.297262+0000","last_became_active":"2024-03-07T07:19:16.301033+0000","last_became_peered":"2024-03-07T07:19:16.301033+0000","last_unstale":"2024-03-07T07:19:17.297262+0000","last_undegraded":"2024-03-07T07:19:17.297262+0000","last_fullsized":"2024-03-07T07:19:17.297262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:34.845258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009648+0000","last_change":"2024-03-07T07:19:16.296414+0000","last_active":"2024-03-07T07:19:18.009648+0000","last_peered":"2024-03-07T07:19:18.009648+0000","last_clean":"2024-03-07T07:19:18.009648+0000","last_became_active":"2024-03-07T07:19:16.296078+0000","last_became_peered":"2024-03-07T07:19:16.296078+0000","last_unstale":"2024-03-07T07:19:18.009648+0000","last_undegraded":"2024-03-07T07:19:18.009648+0000","last_fullsized":"2024-03-07T07:19:18.009648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870192+0000","last_change":"2024-03-07T07:19:16.298815+0000","last_active":"2024-03-07T07:19:17.870192+0000","last_peered":"2024-03-07T07:19:17.870192+0000","last_clean":"2024-03-07T07:19:17.870192+0000","last_became_active":"2024-03-07T07:19:16.298560+0000","last_became_peered":"2024-03-07T07:19:16.298560+0000","last_unstale":"2024-03-07T07:19:17.870192+0000","last_undegraded":"2024-03-07T07:19:17.870192+0000","last_fullsized":"2024-03-07T07:19:17.870192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:14:50.701779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293979+0000","last_change":"2024-03-07T07:19:16.304915+0000","last_active":"2024-03-07T07:19:17.293979+0000","last_peered":"2024-03-07T07:19:17.293979+0000","last_clean":"2024-03-07T07:19:17.293979+0000","last_became_active":"2024-03-07T07:19:16.304581+0000","last_became_peered":"2024-03-07T07:19:16.304581+0000","last_unstale":"2024-03-07T07:19:17.293979+0000","last_undegraded":"2024-03-07T07:19:17.293979+0000","last_fullsized":"2024-03-07T07:19:17.293979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417242+0000","last_change":"2024-03-07T07:19:16.312660+0000","last_active":"2024-03-07T07:19:17.417242+0000","last_peered":"2024-03-07T07:19:17.417242+0000","last_clean":"2024-03-07T07:19:17.417242+0000","last_became_active":"2024-03-07T07:19:16.312356+0000","last_became_peered":"2024-03-07T07:19:16.312356+0000","last_unstale":"2024-03-07T07:19:17.417242+0000","last_undegraded":"2024-03-07T07:19:17.417242+0000","last_fullsized":"2024-03-07T07:19:17.417242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417697+0000","last_change":"2024-03-07T07:19:16.311126+0000","last_active":"2024-03-07T07:19:17.417697+0000","last_peered":"2024-03-07T07:19:17.417697+0000","last_clean":"2024-03-07T07:19:17.417697+0000","last_became_active":"2024-03-07T07:19:16.310806+0000","last_became_peered":"2024-03-07T07:19:16.310806+0000","last_unstale":"2024-03-07T07:19:17.417697+0000","last_undegraded":"2024-03-07T07:19:17.417697+0000","last_fullsized":"2024-03-07T07:19:17.417697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068899+0000","last_change":"2024-03-07T07:19:16.296921+0000","last_active":"2024-03-07T07:19:18.068899+0000","last_peered":"2024-03-07T07:19:18.068899+0000","last_clean":"2024-03-07T07:19:18.068899+0000","last_became_active":"2024-03-07T07:19:16.296657+0000","last_became_peered":"2024-03-07T07:19:16.296657+0000","last_unstale":"2024-03-07T07:19:18.068899+0000","last_undegraded":"2024-03-07T07:19:18.068899+0000","last_fullsized":"2024-03-07T07:19:18.068899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069320+0000","last_change":"2024-03-07T07:19:16.301924+0000","last_active":"2024-03-07T07:19:18.069320+0000","last_peered":"2024-03-07T07:19:18.069320+0000","last_clean":"2024-03-07T07:19:18.069320+0000","last_became_active":"2024-03-07T07:19:16.301657+0000","last_became_peered":"2024-03-07T07:19:16.301657+0000","last_unstale":"2024-03-07T07:19:18.069320+0000","last_undegraded":"2024-03-07T07:19:18.069320+0000","last_fullsized":"2024-03-07T07:19:18.069320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:16:48.547021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296490+0000","last_change":"2024-03-07T07:19:16.300281+0000","last_active":"2024-03-07T07:19:17.296490+0000","last_peered":"2024-03-07T07:19:17.296490+0000","last_clean":"2024-03-07T07:19:17.296490+0000","last_became_active":"2024-03-07T07:19:16.299775+0000","last_became_peered":"2024-03-07T07:19:16.299775+0000","last_unstale":"2024-03-07T07:19:17.296490+0000","last_undegraded":"2024-03-07T07:19:17.296490+0000","last_fullsized":"2024-03-07T07:19:17.296490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870510+0000","last_change":"2024-03-07T07:19:16.296327+0000","last_active":"2024-03-07T07:19:17.870510+0000","last_peered":"2024-03-07T07:19:17.870510+0000","last_clean":"2024-03-07T07:19:17.870510+0000","last_became_active":"2024-03-07T07:19:16.295959+0000","last_became_peered":"2024-03-07T07:19:16.295959+0000","last_unstale":"2024-03-07T07:19:17.870510+0000","last_undegraded":"2024-03-07T07:19:17.870510+0000","last_fullsized":"2024-03-07T07:19:17.870510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417371+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:17.417371+0000","last_peered":"2024-03-07T07:19:17.417371+0000","last_clean":"2024-03-07T07:19:17.417371+0000","last_became_active":"2024-03-07T07:19:16.307389+0000","last_became_peered":"2024-03-07T07:19:16.307389+0000","last_unstale":"2024-03-07T07:19:17.417371+0000","last_undegraded":"2024-03-07T07:19:17.417371+0000","last_fullsized":"2024-03-07T07:19:17.417371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009458+0000","last_change":"2024-03-07T07:19:16.295700+0000","last_active":"2024-03-07T07:19:18.009458+0000","last_peered":"2024-03-07T07:19:18.009458+0000","last_clean":"2024-03-07T07:19:18.009458+0000","last_became_active":"2024-03-07T07:19:16.295428+0000","last_became_peered":"2024-03-07T07:19:16.295428+0000","last_unstale":"2024-03-07T07:19:18.009458+0000","last_undegraded":"2024-03-07T07:19:18.009458+0000","last_fullsized":"2024-03-07T07:19:18.009458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417302+0000","last_change":"2024-03-07T07:19:16.305382+0000","last_active":"2024-03-07T07:19:17.417302+0000","last_peered":"2024-03-07T07:19:17.417302+0000","last_clean":"2024-03-07T07:19:17.417302+0000","last_became_active":"2024-03-07T07:19:16.304697+0000","last_became_peered":"2024-03-07T07:19:16.304697+0000","last_unstale":"2024-03-07T07:19:17.417302+0000","last_undegraded":"2024-03-07T07:19:17.417302+0000","last_fullsized":"2024-03-07T07:19:17.417302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009833+0000","last_change":"2024-03-07T07:19:16.295728+0000","last_active":"2024-03-07T07:19:18.009833+0000","last_peered":"2024-03-07T07:19:18.009833+0000","last_clean":"2024-03-07T07:19:18.009833+0000","last_became_active":"2024-03-07T07:19:16.295472+0000","last_became_peered":"2024-03-07T07:19:16.295472+0000","last_unstale":"2024-03-07T07:19:18.009833+0000","last_undegraded":"2024-03-07T07:19:18.009833+0000","last_fullsized":"2024-03-07T07:19:18.009833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296848+0000","last_change":"2024-03-07T07:19:16.297196+0000","last_active":"2024-03-07T07:19:17.296848+0000","last_peered":"2024-03-07T07:19:17.296848+0000","last_clean":"2024-03-07T07:19:17.296848+0000","last_became_active":"2024-03-07T07:19:16.296727+0000","last_became_peered":"2024-03-07T07:19:16.296727+0000","last_unstale":"2024-03-07T07:19:17.296848+0000","last_undegraded":"2024-03-07T07:19:17.296848+0000","last_fullsized":"2024-03-07T07:19:17.296848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294209+0000","last_change":"2024-03-07T07:19:16.299086+0000","last_active":"2024-03-07T07:19:17.294209+0000","last_peered":"2024-03-07T07:19:17.294209+0000","last_clean":"2024-03-07T07:19:17.294209+0000","last_became_active":"2024-03-07T07:19:16.298819+0000","last_became_peered":"2024-03-07T07:19:16.298819+0000","last_unstale":"2024-03-07T07:19:17.294209+0000","last_undegraded":"2024-03-07T07:19:17.294209+0000","last_fullsized":"2024-03-07T07:19:17.294209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009756+0000","last_change":"2024-03-07T07:19:16.299038+0000","last_active":"2024-03-07T07:19:18.009756+0000","last_peered":"2024-03-07T07:19:18.009756+0000","last_clean":"2024-03-07T07:19:18.009756+0000","last_became_active":"2024-03-07T07:19:16.298748+0000","last_became_peered":"2024-03-07T07:19:16.298748+0000","last_unstale":"2024-03-07T07:19:18.009756+0000","last_undegraded":"2024-03-07T07:19:18.009756+0000","last_fullsized":"2024-03-07T07:19:18.009756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870540+0000","last_change":"2024-03-07T07:19:16.293023+0000","last_active":"2024-03-07T07:19:17.870540+0000","last_peered":"2024-03-07T07:19:17.870540+0000","last_clean":"2024-03-07T07:19:17.870540+0000","last_became_active":"2024-03-07T07:19:16.292670+0000","last_became_peered":"2024-03-07T07:19:16.292670+0000","last_unstale":"2024-03-07T07:19:17.870540+0000","last_undegraded":"2024-03-07T07:19:17.870540+0000","last_fullsized":"2024-03-07T07:19:17.870540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:07.376472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417541+0000","last_change":"2024-03-07T07:19:16.310502+0000","last_active":"2024-03-07T07:19:17.417541+0000","last_peered":"2024-03-07T07:19:17.417541+0000","last_clean":"2024-03-07T07:19:17.417541+0000","last_became_active":"2024-03-07T07:19:16.309883+0000","last_became_peered":"2024-03-07T07:19:16.309883+0000","last_unstale":"2024-03-07T07:19:17.417541+0000","last_undegraded":"2024-03-07T07:19:17.417541+0000","last_fullsized":"2024-03-07T07:19:17.417541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068772+0000","last_change":"2024-03-07T07:19:16.306042+0000","last_active":"2024-03-07T07:19:18.068772+0000","last_peered":"2024-03-07T07:19:18.068772+0000","last_clean":"2024-03-07T07:19:18.068772+0000","last_became_active":"2024-03-07T07:19:16.304954+0000","last_became_peered":"2024-03-07T07:19:16.304954+0000","last_unstale":"2024-03-07T07:19:18.068772+0000","last_undegraded":"2024-03-07T07:19:18.068772+0000","last_fullsized":"2024-03-07T07:19:18.068772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296496+0000","last_change":"2024-03-07T07:19:16.300717+0000","last_active":"2024-03-07T07:19:17.296496+0000","last_peered":"2024-03-07T07:19:17.296496+0000","last_clean":"2024-03-07T07:19:17.296496+0000","last_became_active":"2024-03-07T07:19:16.300170+0000","last_became_peered":"2024-03-07T07:19:16.300170+0000","last_unstale":"2024-03-07T07:19:17.296496+0000","last_undegraded":"2024-03-07T07:19:17.296496+0000","last_fullsized":"2024-03-07T07:19:17.296496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297170+0000","last_change":"2024-03-07T07:19:16.299268+0000","last_active":"2024-03-07T07:19:17.297170+0000","last_peered":"2024-03-07T07:19:17.297170+0000","last_clean":"2024-03-07T07:19:17.297170+0000","last_became_active":"2024-03-07T07:19:16.298987+0000","last_became_peered":"2024-03-07T07:19:16.298987+0000","last_unstale":"2024-03-07T07:19:17.297170+0000","last_undegraded":"2024-03-07T07:19:17.297170+0000","last_fullsized":"2024-03-07T07:19:17.297170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:19.940197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009780+0000","last_change":"2024-03-07T07:19:15.298629+0000","last_active":"2024-03-07T07:19:18.009780+0000","last_peered":"2024-03-07T07:19:18.009780+0000","last_clean":"2024-03-07T07:19:18.009780+0000","last_became_active":"2024-03-07T07:19:15.298338+0000","last_became_peered":"2024-03-07T07:19:15.298338+0000","last_unstale":"2024-03-07T07:19:18.009780+0000","last_undegraded":"2024-03-07T07:19:18.009780+0000","last_fullsized":"2024-03-07T07:19:18.009780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870658+0000","last_change":"2024-03-07T07:19:16.299764+0000","last_active":"2024-03-07T07:19:17.870658+0000","last_peered":"2024-03-07T07:19:17.870658+0000","last_clean":"2024-03-07T07:19:17.870658+0000","last_became_active":"2024-03-07T07:19:16.299493+0000","last_became_peered":"2024-03-07T07:19:16.299493+0000","last_unstale":"2024-03-07T07:19:17.870658+0000","last_undegraded":"2024-03-07T07:19:17.870658+0000","last_fullsized":"2024-03-07T07:19:17.870658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296626+0000","last_change":"2024-03-07T07:19:15.285743+0000","last_active":"2024-03-07T07:19:17.296626+0000","last_peered":"2024-03-07T07:19:17.296626+0000","last_clean":"2024-03-07T07:19:17.296626+0000","last_became_active":"2024-03-07T07:19:15.285484+0000","last_became_peered":"2024-03-07T07:19:15.285484+0000","last_unstale":"2024-03-07T07:19:17.296626+0000","last_undegraded":"2024-03-07T07:19:17.296626+0000","last_fullsized":"2024-03-07T07:19:17.296626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009544+0000","last_change":"2024-03-07T07:19:15.294301+0000","last_active":"2024-03-07T07:19:18.009544+0000","last_peered":"2024-03-07T07:19:18.009544+0000","last_clean":"2024-03-07T07:19:18.009544+0000","last_became_active":"2024-03-07T07:19:15.294023+0000","last_became_peered":"2024-03-07T07:19:15.294023+0000","last_unstale":"2024-03-07T07:19:18.009544+0000","last_undegraded":"2024-03-07T07:19:18.009544+0000","last_fullsized":"2024-03-07T07:19:18.009544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009524+0000","last_change":"2024-03-07T07:19:16.299031+0000","last_active":"2024-03-07T07:19:18.009524+0000","last_peered":"2024-03-07T07:19:18.009524+0000","last_clean":"2024-03-07T07:19:18.009524+0000","last_became_active":"2024-03-07T07:19:16.298760+0000","last_became_peered":"2024-03-07T07:19:16.298760+0000","last_unstale":"2024-03-07T07:19:18.009524+0000","last_undegraded":"2024-03-07T07:19:18.009524+0000","last_fullsized":"2024-03-07T07:19:18.009524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417339+0000","last_change":"2024-03-07T07:19:15.292620+0000","last_active":"2024-03-07T07:19:17.417339+0000","last_peered":"2024-03-07T07:19:17.417339+0000","last_clean":"2024-03-07T07:19:17.417339+0000","last_became_active":"2024-03-07T07:19:15.292337+0000","last_became_peered":"2024-03-07T07:19:15.292337+0000","last_unstale":"2024-03-07T07:19:17.417339+0000","last_undegraded":"2024-03-07T07:19:17.417339+0000","last_fullsized":"2024-03-07T07:19:17.417339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.293989+0000","last_change":"2024-03-07T07:19:16.297970+0000","last_active":"2024-03-07T07:19:17.293989+0000","last_peered":"2024-03-07T07:19:17.293989+0000","last_clean":"2024-03-07T07:19:17.293989+0000","last_became_active":"2024-03-07T07:19:16.297661+0000","last_became_peered":"2024-03-07T07:19:16.297661+0000","last_unstale":"2024-03-07T07:19:17.293989+0000","last_undegraded":"2024-03-07T07:19:17.293989+0000","last_fullsized":"2024-03-07T07:19:17.293989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296514+0000","last_change":"2024-03-07T07:19:15.295817+0000","last_active":"2024-03-07T07:19:17.296514+0000","last_peered":"2024-03-07T07:19:17.296514+0000","last_clean":"2024-03-07T07:19:17.296514+0000","last_became_active":"2024-03-07T07:19:15.295538+0000","last_became_peered":"2024-03-07T07:19:15.295538+0000","last_unstale":"2024-03-07T07:19:17.296514+0000","last_undegraded":"2024-03-07T07:19:17.296514+0000","last_fullsized":"2024-03-07T07:19:17.296514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417710+0000","last_change":"2024-03-07T07:19:16.307152+0000","last_active":"2024-03-07T07:19:17.417710+0000","last_peered":"2024-03-07T07:19:17.417710+0000","last_clean":"2024-03-07T07:19:17.417710+0000","last_became_active":"2024-03-07T07:19:16.306890+0000","last_became_peered":"2024-03-07T07:19:16.306890+0000","last_unstale":"2024-03-07T07:19:17.417710+0000","last_undegraded":"2024-03-07T07:19:17.417710+0000","last_fullsized":"2024-03-07T07:19:17.417710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:48.109405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417582+0000","last_change":"2024-03-07T07:19:15.301303+0000","last_active":"2024-03-07T07:19:17.417582+0000","last_peered":"2024-03-07T07:19:17.417582+0000","last_clean":"2024-03-07T07:19:17.417582+0000","last_became_active":"2024-03-07T07:19:15.301031+0000","last_became_peered":"2024-03-07T07:19:15.301031+0000","last_unstale":"2024-03-07T07:19:17.417582+0000","last_undegraded":"2024-03-07T07:19:17.417582+0000","last_fullsized":"2024-03-07T07:19:17.417582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870348+0000","last_change":"2024-03-07T07:19:16.305499+0000","last_active":"2024-03-07T07:19:17.870348+0000","last_peered":"2024-03-07T07:19:17.870348+0000","last_clean":"2024-03-07T07:19:17.870348+0000","last_became_active":"2024-03-07T07:19:16.304971+0000","last_became_peered":"2024-03-07T07:19:16.304971+0000","last_unstale":"2024-03-07T07:19:17.870348+0000","last_undegraded":"2024-03-07T07:19:17.870348+0000","last_fullsized":"2024-03-07T07:19:17.870348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294183+0000","last_change":"2024-03-07T07:19:15.293364+0000","last_active":"2024-03-07T07:19:17.294183+0000","last_peered":"2024-03-07T07:19:17.294183+0000","last_clean":"2024-03-07T07:19:17.294183+0000","last_became_active":"2024-03-07T07:19:15.293069+0000","last_became_peered":"2024-03-07T07:19:15.293069+0000","last_unstale":"2024-03-07T07:19:17.294183+0000","last_undegraded":"2024-03-07T07:19:17.294183+0000","last_fullsized":"2024-03-07T07:19:17.294183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297511+0000","last_change":"2024-03-07T07:19:16.304722+0000","last_active":"2024-03-07T07:19:17.297511+0000","last_peered":"2024-03-07T07:19:17.297511+0000","last_clean":"2024-03-07T07:19:17.297511+0000","last_became_active":"2024-03-07T07:19:16.304438+0000","last_became_peered":"2024-03-07T07:19:16.304438+0000","last_unstale":"2024-03-07T07:19:17.297511+0000","last_undegraded":"2024-03-07T07:19:17.297511+0000","last_fullsized":"2024-03-07T07:19:17.297511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:24:00.221826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296596+0000","last_change":"2024-03-07T07:19:15.292852+0000","last_active":"2024-03-07T07:19:17.296596+0000","last_peered":"2024-03-07T07:19:17.296596+0000","last_clean":"2024-03-07T07:19:17.296596+0000","last_became_active":"2024-03-07T07:19:15.292584+0000","last_became_peered":"2024-03-07T07:19:15.292584+0000","last_unstale":"2024-03-07T07:19:17.296596+0000","last_undegraded":"2024-03-07T07:19:17.296596+0000","last_fullsized":"2024-03-07T07:19:17.296596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068903+0000","last_change":"2024-03-07T07:19:16.303347+0000","last_active":"2024-03-07T07:19:18.068903+0000","last_peered":"2024-03-07T07:19:18.068903+0000","last_clean":"2024-03-07T07:19:18.068903+0000","last_became_active":"2024-03-07T07:19:16.303083+0000","last_became_peered":"2024-03-07T07:19:16.303083+0000","last_unstale":"2024-03-07T07:19:18.068903+0000","last_undegraded":"2024-03-07T07:19:18.068903+0000","last_fullsized":"2024-03-07T07:19:18.068903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870639+0000","last_change":"2024-03-07T07:19:15.296780+0000","last_active":"2024-03-07T07:19:17.870639+0000","last_peered":"2024-03-07T07:19:17.870639+0000","last_clean":"2024-03-07T07:19:17.870639+0000","last_became_active":"2024-03-07T07:19:15.296300+0000","last_became_peered":"2024-03-07T07:19:15.296300+0000","last_unstale":"2024-03-07T07:19:17.870639+0000","last_undegraded":"2024-03-07T07:19:17.870639+0000","last_fullsized":"2024-03-07T07:19:17.870639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:53:48.349426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870345+0000","last_change":"2024-03-07T07:19:16.297140+0000","last_active":"2024-03-07T07:19:17.870345+0000","last_peered":"2024-03-07T07:19:17.870345+0000","last_clean":"2024-03-07T07:19:17.870345+0000","last_became_active":"2024-03-07T07:19:16.296854+0000","last_became_peered":"2024-03-07T07:19:16.296854+0000","last_unstale":"2024-03-07T07:19:17.870345+0000","last_undegraded":"2024-03-07T07:19:17.870345+0000","last_fullsized":"2024-03-07T07:19:17.870345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417473+0000","last_change":"2024-03-07T07:19:15.299975+0000","last_active":"2024-03-07T07:19:17.417473+0000","last_peered":"2024-03-07T07:19:17.417473+0000","last_clean":"2024-03-07T07:19:17.417473+0000","last_became_active":"2024-03-07T07:19:15.299753+0000","last_became_peered":"2024-03-07T07:19:15.299753+0000","last_unstale":"2024-03-07T07:19:17.417473+0000","last_undegraded":"2024-03-07T07:19:17.417473+0000","last_fullsized":"2024-03-07T07:19:17.417473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068921+0000","last_change":"2024-03-07T07:19:16.307214+0000","last_active":"2024-03-07T07:19:18.068921+0000","last_peered":"2024-03-07T07:19:18.068921+0000","last_clean":"2024-03-07T07:19:18.068921+0000","last_became_active":"2024-03-07T07:19:16.306690+0000","last_became_peered":"2024-03-07T07:19:16.306690+0000","last_unstale":"2024-03-07T07:19:18.068921+0000","last_undegraded":"2024-03-07T07:19:18.068921+0000","last_fullsized":"2024-03-07T07:19:18.068921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:55.537688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296268+0000","last_change":"2024-03-07T07:19:15.286950+0000","last_active":"2024-03-07T07:19:17.296268+0000","last_peered":"2024-03-07T07:19:17.296268+0000","last_clean":"2024-03-07T07:19:17.296268+0000","last_became_active":"2024-03-07T07:19:15.286655+0000","last_became_peered":"2024-03-07T07:19:15.286655+0000","last_unstale":"2024-03-07T07:19:17.296268+0000","last_undegraded":"2024-03-07T07:19:17.296268+0000","last_fullsized":"2024-03-07T07:19:17.296268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296267+0000","last_change":"2024-03-07T07:19:16.299235+0000","last_active":"2024-03-07T07:19:17.296267+0000","last_peered":"2024-03-07T07:19:17.296267+0000","last_clean":"2024-03-07T07:19:17.296267+0000","last_became_active":"2024-03-07T07:19:16.298945+0000","last_became_peered":"2024-03-07T07:19:16.298945+0000","last_unstale":"2024-03-07T07:19:17.296267+0000","last_undegraded":"2024-03-07T07:19:17.296267+0000","last_fullsized":"2024-03-07T07:19:17.296267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068707+0000","last_change":"2024-03-07T07:19:15.289721+0000","last_active":"2024-03-07T07:19:18.068707+0000","last_peered":"2024-03-07T07:19:18.068707+0000","last_clean":"2024-03-07T07:19:18.068707+0000","last_became_active":"2024-03-07T07:19:15.289410+0000","last_became_peered":"2024-03-07T07:19:15.289410+0000","last_unstale":"2024-03-07T07:19:18.068707+0000","last_undegraded":"2024-03-07T07:19:18.068707+0000","last_fullsized":"2024-03-07T07:19:18.068707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068748+0000","last_change":"2024-03-07T07:19:16.306954+0000","last_active":"2024-03-07T07:19:18.068748+0000","last_peered":"2024-03-07T07:19:18.068748+0000","last_clean":"2024-03-07T07:19:18.068748+0000","last_became_active":"2024-03-07T07:19:16.306639+0000","last_became_peered":"2024-03-07T07:19:16.306639+0000","last_unstale":"2024-03-07T07:19:18.068748+0000","last_undegraded":"2024-03-07T07:19:18.068748+0000","last_fullsized":"2024-03-07T07:19:18.068748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297392+0000","last_change":"2024-03-07T07:19:15.297356+0000","last_active":"2024-03-07T07:19:17.297392+0000","last_peered":"2024-03-07T07:19:17.297392+0000","last_clean":"2024-03-07T07:19:17.297392+0000","last_became_active":"2024-03-07T07:19:15.296989+0000","last_became_peered":"2024-03-07T07:19:15.296989+0000","last_unstale":"2024-03-07T07:19:17.297392+0000","last_undegraded":"2024-03-07T07:19:17.297392+0000","last_fullsized":"2024-03-07T07:19:17.297392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:17:57.790720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069126+0000","last_change":"2024-03-07T07:19:16.306845+0000","last_active":"2024-03-07T07:19:18.069126+0000","last_peered":"2024-03-07T07:19:18.069126+0000","last_clean":"2024-03-07T07:19:18.069126+0000","last_became_active":"2024-03-07T07:19:16.306406+0000","last_became_peered":"2024-03-07T07:19:16.306406+0000","last_unstale":"2024-03-07T07:19:18.069126+0000","last_undegraded":"2024-03-07T07:19:18.069126+0000","last_fullsized":"2024-03-07T07:19:18.069126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:00.501152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068846+0000","last_change":"2024-03-07T07:19:15.293365+0000","last_active":"2024-03-07T07:19:18.068846+0000","last_peered":"2024-03-07T07:19:18.068846+0000","last_clean":"2024-03-07T07:19:18.068846+0000","last_became_active":"2024-03-07T07:19:15.293062+0000","last_became_peered":"2024-03-07T07:19:15.293062+0000","last_unstale":"2024-03-07T07:19:18.068846+0000","last_undegraded":"2024-03-07T07:19:18.068846+0000","last_fullsized":"2024-03-07T07:19:18.068846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:44:40.592736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069160+0000","last_change":"2024-03-07T07:19:16.307514+0000","last_active":"2024-03-07T07:19:18.069160+0000","last_peered":"2024-03-07T07:19:18.069160+0000","last_clean":"2024-03-07T07:19:18.069160+0000","last_became_active":"2024-03-07T07:19:16.307226+0000","last_became_peered":"2024-03-07T07:19:16.307226+0000","last_unstale":"2024-03-07T07:19:18.069160+0000","last_undegraded":"2024-03-07T07:19:18.069160+0000","last_fullsized":"2024-03-07T07:19:18.069160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:25:12.269915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294284+0000","last_change":"2024-03-07T07:19:15.290118+0000","last_active":"2024-03-07T07:19:17.294284+0000","last_peered":"2024-03-07T07:19:17.294284+0000","last_clean":"2024-03-07T07:19:17.294284+0000","last_became_active":"2024-03-07T07:19:15.289812+0000","last_became_peered":"2024-03-07T07:19:15.289812+0000","last_unstale":"2024-03-07T07:19:17.294284+0000","last_undegraded":"2024-03-07T07:19:17.294284+0000","last_fullsized":"2024-03-07T07:19:17.294284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417670+0000","last_change":"2024-03-07T07:19:16.303739+0000","last_active":"2024-03-07T07:19:17.417670+0000","last_peered":"2024-03-07T07:19:17.417670+0000","last_clean":"2024-03-07T07:19:17.417670+0000","last_became_active":"2024-03-07T07:19:16.303279+0000","last_became_peered":"2024-03-07T07:19:16.303279+0000","last_unstale":"2024-03-07T07:19:17.417670+0000","last_undegraded":"2024-03-07T07:19:17.417670+0000","last_fullsized":"2024-03-07T07:19:17.417670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:59:14.419772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068866+0000","last_change":"2024-03-07T07:19:15.296198+0000","last_active":"2024-03-07T07:19:18.068866+0000","last_peered":"2024-03-07T07:19:18.068866+0000","last_clean":"2024-03-07T07:19:18.068866+0000","last_became_active":"2024-03-07T07:19:15.295886+0000","last_became_peered":"2024-03-07T07:19:15.295886+0000","last_unstale":"2024-03-07T07:19:18.068866+0000","last_undegraded":"2024-03-07T07:19:18.068866+0000","last_fullsized":"2024-03-07T07:19:18.068866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:19:18.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069075+0000","last_change":"2024-03-07T07:19:16.307657+0000","last_active":"2024-03-07T07:19:18.069075+0000","last_peered":"2024-03-07T07:19:18.069075+0000","last_clean":"2024-03-07T07:19:18.069075+0000","last_became_active":"2024-03-07T07:19:16.307119+0000","last_became_peered":"2024-03-07T07:19:16.307119+0000","last_unstale":"2024-03-07T07:19:18.069075+0000","last_undegraded":"2024-03-07T07:19:18.069075+0000","last_fullsized":"2024-03-07T07:19:18.069075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068695+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:19:18.068695+0000","last_peered":"2024-03-07T07:19:18.068695+0000","last_clean":"2024-03-07T07:19:18.068695+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:19:18.068695+0000","last_undegraded":"2024-03-07T07:19:18.068695+0000","last_fullsized":"2024-03-07T07:19:18.068695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417348+0000","last_change":"2024-03-07T07:19:15.295100+0000","last_active":"2024-03-07T07:19:17.417348+0000","last_peered":"2024-03-07T07:19:17.417348+0000","last_clean":"2024-03-07T07:19:17.417348+0000","last_became_active":"2024-03-07T07:19:15.294643+0000","last_became_peered":"2024-03-07T07:19:15.294643+0000","last_unstale":"2024-03-07T07:19:17.417348+0000","last_undegraded":"2024-03-07T07:19:17.417348+0000","last_fullsized":"2024-03-07T07:19:17.417348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.306837+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.305695+0000","last_became_peered":"2024-03-07T07:19:16.305695+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:20:08.402557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068642+0000","last_change":"2024-03-07T07:19:15.301442+0000","last_active":"2024-03-07T07:19:18.068642+0000","last_peered":"2024-03-07T07:19:18.068642+0000","last_clean":"2024-03-07T07:19:18.068642+0000","last_became_active":"2024-03-07T07:19:15.300526+0000","last_became_peered":"2024-03-07T07:19:15.300526+0000","last_unstale":"2024-03-07T07:19:18.068642+0000","last_undegraded":"2024-03-07T07:19:18.068642+0000","last_fullsized":"2024-03-07T07:19:18.068642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:52:33.839717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870083+0000","last_change":"2024-03-07T07:19:16.297399+0000","last_active":"2024-03-07T07:19:17.870083+0000","last_peered":"2024-03-07T07:19:17.870083+0000","last_clean":"2024-03-07T07:19:17.870083+0000","last_became_active":"2024-03-07T07:19:16.296941+0000","last_became_peered":"2024-03-07T07:19:16.296941+0000","last_unstale":"2024-03-07T07:19:17.870083+0000","last_undegraded":"2024-03-07T07:19:17.870083+0000","last_fullsized":"2024-03-07T07:19:17.870083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:10.125663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068824+0000","last_change":"2024-03-07T07:19:15.291261+0000","last_active":"2024-03-07T07:19:18.068824+0000","last_peered":"2024-03-07T07:19:18.068824+0000","last_clean":"2024-03-07T07:19:18.068824+0000","last_became_active":"2024-03-07T07:19:15.290977+0000","last_became_peered":"2024-03-07T07:19:15.290977+0000","last_unstale":"2024-03-07T07:19:18.068824+0000","last_undegraded":"2024-03-07T07:19:18.068824+0000","last_fullsized":"2024-03-07T07:19:18.068824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:25:02.313644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294009+0000","last_change":"2024-03-07T07:19:16.298068+0000","last_active":"2024-03-07T07:19:17.294009+0000","last_peered":"2024-03-07T07:19:17.294009+0000","last_clean":"2024-03-07T07:19:17.294009+0000","last_became_active":"2024-03-07T07:19:16.297795+0000","last_became_peered":"2024-03-07T07:19:16.297795+0000","last_unstale":"2024-03-07T07:19:17.294009+0000","last_undegraded":"2024-03-07T07:19:17.294009+0000","last_fullsized":"2024-03-07T07:19:17.294009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:35:16.128284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417588+0000","last_change":"2024-03-07T07:19:15.296795+0000","last_active":"2024-03-07T07:19:17.417588+0000","last_peered":"2024-03-07T07:19:17.417588+0000","last_clean":"2024-03-07T07:19:17.417588+0000","last_became_active":"2024-03-07T07:19:15.296534+0000","last_became_peered":"2024-03-07T07:19:15.296534+0000","last_unstale":"2024-03-07T07:19:17.417588+0000","last_undegraded":"2024-03-07T07:19:17.417588+0000","last_fullsized":"2024-03-07T07:19:17.417588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297568+0000","last_change":"2024-03-07T07:19:16.295673+0000","last_active":"2024-03-07T07:19:17.297568+0000","last_peered":"2024-03-07T07:19:17.297568+0000","last_clean":"2024-03-07T07:19:17.297568+0000","last_became_active":"2024-03-07T07:19:16.295139+0000","last_became_peered":"2024-03-07T07:19:16.295139+0000","last_unstale":"2024-03-07T07:19:17.297568+0000","last_undegraded":"2024-03-07T07:19:17.297568+0000","last_fullsized":"2024-03-07T07:19:17.297568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:28.867899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294148+0000","last_change":"2024-03-07T07:19:15.297274+0000","last_active":"2024-03-07T07:19:17.294148+0000","last_peered":"2024-03-07T07:19:17.294148+0000","last_clean":"2024-03-07T07:19:17.294148+0000","last_became_active":"2024-03-07T07:19:15.297000+0000","last_became_peered":"2024-03-07T07:19:15.297000+0000","last_unstale":"2024-03-07T07:19:17.294148+0000","last_undegraded":"2024-03-07T07:19:17.294148+0000","last_fullsized":"2024-03-07T07:19:17.294148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417617+0000","last_change":"2024-03-07T07:19:16.305048+0000","last_active":"2024-03-07T07:19:17.417617+0000","last_peered":"2024-03-07T07:19:17.417617+0000","last_clean":"2024-03-07T07:19:17.417617+0000","last_became_active":"2024-03-07T07:19:16.303993+0000","last_became_peered":"2024-03-07T07:19:16.303993+0000","last_unstale":"2024-03-07T07:19:17.417617+0000","last_undegraded":"2024-03-07T07:19:17.417617+0000","last_fullsized":"2024-03-07T07:19:17.417617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:36:14.027754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069397+0000","last_change":"2024-03-07T07:19:15.300909+0000","last_active":"2024-03-07T07:19:18.069397+0000","last_peered":"2024-03-07T07:19:18.069397+0000","last_clean":"2024-03-07T07:19:18.069397+0000","last_became_active":"2024-03-07T07:19:15.300573+0000","last_became_peered":"2024-03-07T07:19:15.300573+0000","last_unstale":"2024-03-07T07:19:18.069397+0000","last_undegraded":"2024-03-07T07:19:18.069397+0000","last_fullsized":"2024-03-07T07:19:18.069397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:57:36.412205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294190+0000","last_change":"2024-03-07T07:19:16.298525+0000","last_active":"2024-03-07T07:19:17.294190+0000","last_peered":"2024-03-07T07:19:17.294190+0000","last_clean":"2024-03-07T07:19:17.294190+0000","last_became_active":"2024-03-07T07:19:16.298264+0000","last_became_peered":"2024-03-07T07:19:16.298264+0000","last_unstale":"2024-03-07T07:19:17.294190+0000","last_undegraded":"2024-03-07T07:19:17.294190+0000","last_fullsized":"2024-03-07T07:19:17.294190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:33.724824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.295564+0000","last_change":"2024-03-07T07:19:15.291265+0000","last_active":"2024-03-07T07:19:17.295564+0000","last_peered":"2024-03-07T07:19:17.295564+0000","last_clean":"2024-03-07T07:19:17.295564+0000","last_became_active":"2024-03-07T07:19:15.290889+0000","last_became_peered":"2024-03-07T07:19:15.290889+0000","last_unstale":"2024-03-07T07:19:17.295564+0000","last_undegraded":"2024-03-07T07:19:17.295564+0000","last_fullsized":"2024-03-07T07:19:17.295564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009905+0000","last_change":"2024-03-07T07:19:16.303881+0000","last_active":"2024-03-07T07:19:18.009905+0000","last_peered":"2024-03-07T07:19:18.009905+0000","last_clean":"2024-03-07T07:19:18.009905+0000","last_became_active":"2024-03-07T07:19:16.303597+0000","last_became_peered":"2024-03-07T07:19:16.303597+0000","last_unstale":"2024-03-07T07:19:18.009905+0000","last_undegraded":"2024-03-07T07:19:18.009905+0000","last_fullsized":"2024-03-07T07:19:18.009905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296735+0000","last_change":"2024-03-07T07:19:15.296436+0000","last_active":"2024-03-07T07:19:17.296735+0000","last_peered":"2024-03-07T07:19:17.296735+0000","last_clean":"2024-03-07T07:19:17.296735+0000","last_became_active":"2024-03-07T07:19:15.296152+0000","last_became_peered":"2024-03-07T07:19:15.296152+0000","last_unstale":"2024-03-07T07:19:17.296735+0000","last_undegraded":"2024-03-07T07:19:17.296735+0000","last_fullsized":"2024-03-07T07:19:17.296735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870218+0000","last_change":"2024-03-07T07:19:16.300794+0000","last_active":"2024-03-07T07:19:17.870218+0000","last_peered":"2024-03-07T07:19:17.870218+0000","last_clean":"2024-03-07T07:19:17.870218+0000","last_became_active":"2024-03-07T07:19:16.300531+0000","last_became_peered":"2024-03-07T07:19:16.300531+0000","last_unstale":"2024-03-07T07:19:17.870218+0000","last_undegraded":"2024-03-07T07:19:17.870218+0000","last_fullsized":"2024-03-07T07:19:17.870218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:13:38.242681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.068653+0000","last_change":"2024-03-07T07:19:15.301871+0000","last_active":"2024-03-07T07:19:18.068653+0000","last_peered":"2024-03-07T07:19:18.068653+0000","last_clean":"2024-03-07T07:19:18.068653+0000","last_became_active":"2024-03-07T07:19:15.301269+0000","last_became_peered":"2024-03-07T07:19:15.301269+0000","last_unstale":"2024-03-07T07:19:18.068653+0000","last_undegraded":"2024-03-07T07:19:18.068653+0000","last_fullsized":"2024-03-07T07:19:18.068653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.297261+0000","last_change":"2024-03-07T07:19:16.293571+0000","last_active":"2024-03-07T07:19:17.297261+0000","last_peered":"2024-03-07T07:19:17.297261+0000","last_clean":"2024-03-07T07:19:17.297261+0000","last_became_active":"2024-03-07T07:19:16.293217+0000","last_became_peered":"2024-03-07T07:19:16.293217+0000","last_unstale":"2024-03-07T07:19:17.297261+0000","last_undegraded":"2024-03-07T07:19:17.297261+0000","last_fullsized":"2024-03-07T07:19:17.297261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:11.899192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294039+0000","last_change":"2024-03-07T07:19:15.291033+0000","last_active":"2024-03-07T07:19:17.294039+0000","last_peered":"2024-03-07T07:19:17.294039+0000","last_clean":"2024-03-07T07:19:17.294039+0000","last_became_active":"2024-03-07T07:19:15.290699+0000","last_became_peered":"2024-03-07T07:19:15.290699+0000","last_unstale":"2024-03-07T07:19:17.294039+0000","last_undegraded":"2024-03-07T07:19:17.294039+0000","last_fullsized":"2024-03-07T07:19:17.294039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:13:10.893651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009889+0000","last_change":"2024-03-07T07:19:16.300154+0000","last_active":"2024-03-07T07:19:18.009889+0000","last_peered":"2024-03-07T07:19:18.009889+0000","last_clean":"2024-03-07T07:19:18.009889+0000","last_became_active":"2024-03-07T07:19:16.299831+0000","last_became_peered":"2024-03-07T07:19:16.299831+0000","last_unstale":"2024-03-07T07:19:18.009889+0000","last_undegraded":"2024-03-07T07:19:18.009889+0000","last_fullsized":"2024-03-07T07:19:18.009889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:36:50.480912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.294117+0000","last_change":"2024-03-07T07:19:15.292788+0000","last_active":"2024-03-07T07:19:17.294117+0000","last_peered":"2024-03-07T07:19:17.294117+0000","last_clean":"2024-03-07T07:19:17.294117+0000","last_became_active":"2024-03-07T07:19:15.292478+0000","last_became_peered":"2024-03-07T07:19:15.292478+0000","last_unstale":"2024-03-07T07:19:17.294117+0000","last_undegraded":"2024-03-07T07:19:17.294117+0000","last_fullsized":"2024-03-07T07:19:17.294117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009880+0000","last_change":"2024-03-07T07:19:16.302180+0000","last_active":"2024-03-07T07:19:18.009880+0000","last_peered":"2024-03-07T07:19:18.009880+0000","last_clean":"2024-03-07T07:19:18.009880+0000","last_became_active":"2024-03-07T07:19:16.301894+0000","last_became_peered":"2024-03-07T07:19:16.301894+0000","last_unstale":"2024-03-07T07:19:18.009880+0000","last_undegraded":"2024-03-07T07:19:18.009880+0000","last_fullsized":"2024-03-07T07:19:18.009880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:53:07.105252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069079+0000","last_change":"2024-03-07T07:19:15.300817+0000","last_active":"2024-03-07T07:19:18.069079+0000","last_peered":"2024-03-07T07:19:18.069079+0000","last_clean":"2024-03-07T07:19:18.069079+0000","last_became_active":"2024-03-07T07:19:15.300502+0000","last_became_peered":"2024-03-07T07:19:15.300502+0000","last_unstale":"2024-03-07T07:19:18.069079+0000","last_undegraded":"2024-03-07T07:19:18.069079+0000","last_fullsized":"2024-03-07T07:19:18.069079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:12:21.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.296526+0000","last_change":"2024-03-07T07:19:16.300613+0000","last_active":"2024-03-07T07:19:17.296526+0000","last_peered":"2024-03-07T07:19:17.296526+0000","last_clean":"2024-03-07T07:19:17.296526+0000","last_became_active":"2024-03-07T07:19:16.299969+0000","last_became_peered":"2024-03-07T07:19:16.299969+0000","last_unstale":"2024-03-07T07:19:17.296526+0000","last_undegraded":"2024-03-07T07:19:17.296526+0000","last_fullsized":"2024-03-07T07:19:17.296526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:11:48.870287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417649+0000","last_change":"2024-03-07T07:19:16.316583+0000","last_active":"2024-03-07T07:19:17.417649+0000","last_peered":"2024-03-07T07:19:17.417649+0000","last_clean":"2024-03-07T07:19:17.417649+0000","last_became_active":"2024-03-07T07:19:16.316351+0000","last_became_peered":"2024-03-07T07:19:16.316351+0000","last_unstale":"2024-03-07T07:19:17.417649+0000","last_undegraded":"2024-03-07T07:19:17.417649+0000","last_fullsized":"2024-03-07T07:19:17.417649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:39.060613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069099+0000","last_change":"2024-03-07T07:19:15.300879+0000","last_active":"2024-03-07T07:19:18.069099+0000","last_peered":"2024-03-07T07:19:18.069099+0000","last_clean":"2024-03-07T07:19:18.069099+0000","last_became_active":"2024-03-07T07:19:15.300594+0000","last_became_peered":"2024-03-07T07:19:15.300594+0000","last_unstale":"2024-03-07T07:19:18.069099+0000","last_undegraded":"2024-03-07T07:19:18.069099+0000","last_fullsized":"2024-03-07T07:19:18.069099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:03:27.862358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.417463+0000","last_change":"2024-03-07T07:19:16.313560+0000","last_active":"2024-03-07T07:19:17.417463+0000","last_peered":"2024-03-07T07:19:17.417463+0000","last_clean":"2024-03-07T07:19:17.417463+0000","last_became_active":"2024-03-07T07:19:16.313130+0000","last_became_peered":"2024-03-07T07:19:16.313130+0000","last_unstale":"2024-03-07T07:19:17.417463+0000","last_undegraded":"2024-03-07T07:19:17.417463+0000","last_fullsized":"2024-03-07T07:19:17.417463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:52:46.728068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.069186+0000","last_change":"2024-03-07T07:19:15.301563+0000","last_active":"2024-03-07T07:19:18.069186+0000","last_peered":"2024-03-07T07:19:18.069186+0000","last_clean":"2024-03-07T07:19:18.069186+0000","last_became_active":"2024-03-07T07:19:15.301282+0000","last_became_peered":"2024-03-07T07:19:15.301282+0000","last_unstale":"2024-03-07T07:19:18.069186+0000","last_undegraded":"2024-03-07T07:19:18.069186+0000","last_fullsized":"2024-03-07T07:19:18.069186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:23:59.216284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870211+0000","last_change":"2024-03-07T07:19:16.306438+0000","last_active":"2024-03-07T07:19:17.870211+0000","last_peered":"2024-03-07T07:19:17.870211+0000","last_clean":"2024-03-07T07:19:17.870211+0000","last_became_active":"2024-03-07T07:19:16.305209+0000","last_became_peered":"2024-03-07T07:19:16.305209+0000","last_unstale":"2024-03-07T07:19:17.870211+0000","last_undegraded":"2024-03-07T07:19:17.870211+0000","last_fullsized":"2024-03-07T07:19:17.870211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:33:39.054035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009925+0000","last_change":"2024-03-07T07:19:15.299866+0000","last_active":"2024-03-07T07:19:18.009925+0000","last_peered":"2024-03-07T07:19:18.009925+0000","last_clean":"2024-03-07T07:19:18.009925+0000","last_became_active":"2024-03-07T07:19:15.299621+0000","last_became_peered":"2024-03-07T07:19:15.299621+0000","last_unstale":"2024-03-07T07:19:18.009925+0000","last_undegraded":"2024-03-07T07:19:18.009925+0000","last_fullsized":"2024-03-07T07:19:18.009925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:18.009615+0000","last_change":"2024-03-07T07:19:16.296975+0000","last_active":"2024-03-07T07:19:18.009615+0000","last_peered":"2024-03-07T07:19:18.009615+0000","last_clean":"2024-03-07T07:19:18.009615+0000","last_became_active":"2024-03-07T07:19:16.296702+0000","last_became_peered":"2024-03-07T07:19:16.296702+0000","last_unstale":"2024-03-07T07:19:18.009615+0000","last_undegraded":"2024-03-07T07:19:18.009615+0000","last_fullsized":"2024-03-07T07:19:18.009615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:15.273316+0000","last_clean_scrub_stamp":"2024-03-07T07:19:15.273316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:30:28.917584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-07T07:19:17.870446+0000","last_change":"2024-03-07T07:19:15.303925+0000","last_active":"2024-03-07T07:19:17.870446+0000","last_peered":"2024-03-07T07:19:17.870446+0000","last_clean":"2024-03-07T07:19:17.870446+0000","last_became_active":"2024-03-07T07:19:15.303646+0000","last_became_peered":"2024-03-07T07:19:15.303646+0000","last_unstale":"2024-03-07T07:19:17.870446+0000","last_undegraded":"2024-03-07T07:19:17.870446+0000","last_fullsized":"2024-03-07T07:19:17.870446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:14.267775+0000","last_clean_scrub_stamp":"2024-03-07T07:19:14.267775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:27:47.477056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:19:27.142 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-07T07:19:27.142 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-07T07:19:27.142 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-07T07:19:27.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-07T07:19:27.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.279+0000 7f25d3474640 1 Processor -- start 2024-03-07T07:19:27.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.279+0000 7f25d3474640 1 -- start start 2024-03-07T07:19:27.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d3474640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f25cc162030 0x7f25cc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d3474640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f25cc1416d0 0x7f25cc162410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d3474640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25cc141c60 con 0x7f25cc162950 2024-03-07T07:19:27.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d3474640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc0b44b0 con 0x7f25cc1416d0 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d3474640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc15ecd0 con 0x7f25cc162030 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d0a0f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f25cc1416d0 0x7f25cc162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d0a0f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f25cc1416d0 0x7f25cc162410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34598/0 (socket says 172.21.15.64:34598) 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d0a0f640 1 -- 172.21.15.64:0/1773890856 learned_addr learned my addr 172.21.15.64:0/1773890856 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d1210640 1 --2- 172.21.15.64:0/1773890856 >> v2:172.21.15.120:3300/0 conn(0x7f25cc162030 0x7f25cc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d0a0f640 1 -- 172.21.15.64:0/1773890856 >> v2:172.21.15.120:3300/0 conn(0x7f25cc162030 msgr2=0x7f25cc15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d0a0f640 1 --2- 172.21.15.64:0/1773890856 >> v2:172.21.15.120:3300/0 conn(0x7f25cc162030 0x7f25cc15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d0a0f640 1 -- 172.21.15.64:0/1773890856 >> v1:172.21.15.120:6789/0 conn(0x7f25cc162950 legacy=0x7f25cc0d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25c3fff640 1 -- 172.21.15.64:0/1773890856 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766596971 0 0) 0x7f25cc141c60 con 0x7f25cc162950 2024-03-07T07:19:27.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.280+0000 7f25d0a0f640 1 -- 172.21.15.64:0/1773890856 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc15efb0 con 0x7f25cc1416d0 2024-03-07T07:19:27.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.281+0000 7f25d0a0f640 1 --2- 172.21.15.64:0/1773890856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc162410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25cc074d40 tx=0x7f25bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6227c579398dace7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.281+0000 7f25c3fff640 1 -- 172.21.15.64:0/1773890856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc034070 con 0x7f25cc1416d0 2024-03-07T07:19:27.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.281+0000 7f25c3fff640 1 -- 172.21.15.64:0/1773890856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc030860 con 0x7f25cc1416d0 2024-03-07T07:19:27.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.281+0000 7f25c3fff640 1 -- 172.21.15.64:0/1773890856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc030c30 con 0x7f25cc1416d0 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 -- 172.21.15.64:0/1773890856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 msgr2=0x7f25cc162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 --2- 172.21.15.64:0/1773890856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc162410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25cc074d40 tx=0x7f25bc02b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 -- 172.21.15.64:0/1773890856 shutdown_connections 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 --2- 172.21.15.64:0/1773890856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc162410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 --2- 172.21.15.64:0/1773890856 >> v2:172.21.15.120:3300/0 conn(0x7f25cc162030 0x7f25cc15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 -- 172.21.15.64:0/1773890856 >> 172.21.15.64:0/1773890856 conn(0x7f25cc070400 msgr2=0x7f25cc0d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 -- 172.21.15.64:0/1773890856 shutdown_connections 2024-03-07T07:19:27.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.282+0000 7f25d3474640 1 -- 172.21.15.64:0/1773890856 wait complete. 2024-03-07T07:19:27.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.283+0000 7f25d3474640 1 Processor -- start 2024-03-07T07:19:27.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.283+0000 7f25d3474640 1 -- start start 2024-03-07T07:19:27.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d3474640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc13da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d3474640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25cc162030 0x7f25cc1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d3474640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25cc162950 0x7f25cc0d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d3474640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0d3ba0 con 0x7f25cc1416d0 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d3474640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc15f570 con 0x7f25cc162950 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d3474640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0d3d20 con 0x7f25cc162030 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc13da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc13da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34614/0 (socket says 172.21.15.64:34614) 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 -- 172.21.15.64:0/447167909 learned_addr learned my addr 172.21.15.64:0/447167909 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1a11640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25cc162950 0x7f25cc0d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d0a0f640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25cc162030 0x7f25cc1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 -- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25cc162030 msgr2=0x7f25cc1567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25cc162030 0x7f25cc1567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 -- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25cc162950 msgr2=0x7f25cc0d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25cc162950 0x7f25cc0d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 -- 172.21.15.64:0/447167909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc156d30 con 0x7f25cc1416d0 2024-03-07T07:19:27.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.284+0000 7f25d1210640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc13da80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25c400e9e0 tx=0x7f25c400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.285+0000 7f25c1ffb640 1 -- 172.21.15.64:0/447167909 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4017070 con 0x7f25cc1416d0 2024-03-07T07:19:27.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.285+0000 7f25c1ffb640 1 -- 172.21.15.64:0/447167909 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400c9d0 con 0x7f25cc1416d0 2024-03-07T07:19:27.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.285+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc0d3790 con 0x7f25cc1416d0 2024-03-07T07:19:27.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.285+0000 7f25c1ffb640 1 -- 172.21.15.64:0/447167909 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400cdc0 con 0x7f25cc1416d0 2024-03-07T07:19:27.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.285+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc144dc0 con 0x7f25cc1416d0 2024-03-07T07:19:27.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.286+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc062c90 con 0x7f25cc1416d0 2024-03-07T07:19:27.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.286+0000 7f25c1ffb640 1 -- 172.21.15.64:0/447167909 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f25c401a070 con 0x7f25cc1416d0 2024-03-07T07:19:27.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.287+0000 7f25c1ffb640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f25b006b4d0 0x7f25b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.287+0000 7f25c1ffb640 1 -- 172.21.15.64:0/447167909 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25c408a140 con 0x7f25cc1416d0 2024-03-07T07:19:27.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.287+0000 7f25d0a0f640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f25b006b4d0 0x7f25b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.287+0000 7f25d0a0f640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f25b006b4d0 0x7f25b006d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f25bc036000 tx=0x7f25bc03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.291+0000 7f25c1ffb640 1 -- 172.21.15.64:0/447167909 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f25c4058970 con 0x7f25cc1416d0 2024-03-07T07:19:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.509+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f25cc144890 con 0x7f25cc1416d0 2024-03-07T07:19:27.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.509+0000 7f25c1ffb640 1 -- 172.21.15.64:0/447167909 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f25c405ca10 con 0x7f25cc1416d0 2024-03-07T07:19:27.509 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:27.510 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-07T07:19:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f25b006b4d0 msgr2=0x7f25b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f25b006b4d0 0x7f25b006d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f25bc036000 tx=0x7f25bc03c040 comp rx=0 tx=0).stop 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 msgr2=0x7f25cc13da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc13da80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25c400e9e0 tx=0x7f25c400eea0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 shutdown_connections 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f25b006b4d0 0x7f25b006d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25cc162950 0x7f25cc0d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25cc162030 0x7f25cc1567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 --2- 172.21.15.64:0/447167909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25cc1416d0 0x7f25cc13da80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 >> 172.21.15.64:0/447167909 conn(0x7f25cc070400 msgr2=0x7f25cc161180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 shutdown_connections 2024-03-07T07:19:27.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.511+0000 7f25d3474640 1 -- 172.21.15.64:0/447167909 wait complete. 2024-03-07T07:19:27.522 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-07T07:19:27.522 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:27.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.623+0000 7fe6b8bc9640 1 Processor -- start 2024-03-07T07:19:27.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.623+0000 7fe6b8bc9640 1 -- start start 2024-03-07T07:19:27.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b8bc9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b415f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b8bc9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe6b4070400 0x7fe6b40708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b8bc9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b4071130 con 0x7fe6b415e750 2024-03-07T07:19:27.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b8bc9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40b05b0 con 0x7fe6b4070400 2024-03-07T07:19:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b8bc9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4071410 con 0x7fe6b40aec30 2024-03-07T07:19:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b259c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe6b415e750 0x7fe6b415eb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48740/0 (socket says 172.21.15.64:48740) 2024-03-07T07:19:27.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b259c640 1 -- 172.21.15.64:0/1274422129 learned_addr learned my addr 172.21.15.64:0/1274422129 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b2d9d640 1 --2- 172.21.15.64:0/1274422129 >> v2:172.21.15.120:3300/0 conn(0x7fe6b4070400 0x7fe6b40708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.624+0000 7fe6b1d9b640 1 --2- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.625+0000 7fe6b159a640 1 -- 172.21.15.64:0/1274422129 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784557487 0 0) 0x7fe6b4071130 con 0x7fe6b415e750 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.625+0000 7fe6b159a640 1 -- 172.21.15.64:0/1274422129 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a0003520 con 0x7fe6b415e750 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.625+0000 7fe6b2d9d640 1 -- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 msgr2=0x7fe6b415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.625+0000 7fe6b2d9d640 1 --2- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.625+0000 7fe6b2d9d640 1 -- 172.21.15.64:0/1274422129 >> v1:172.21.15.64:6789/0 conn(0x7fe6b415e750 legacy=0x7fe6b415eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.625+0000 7fe6b2d9d640 1 -- 172.21.15.64:0/1274422129 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40716f0 con 0x7fe6b4070400 2024-03-07T07:19:27.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.625+0000 7fe6b1d9b640 1 --2- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b2d9d640 1 --2- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b4070400 0x7fe6b40708b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009d90 tx=0x7fe6a402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=967d8b55be99aec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b159a640 1 -- 172.21.15.64:0/1274422129 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4039070 con 0x7fe6b4070400 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b159a640 1 -- 172.21.15.64:0/1274422129 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a402cc50 con 0x7fe6b4070400 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b159a640 1 -- 172.21.15.64:0/1274422129 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4034440 con 0x7fe6b4070400 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b4070400 msgr2=0x7fe6b40708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b4070400 0x7fe6b40708b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009d90 tx=0x7fe6a402bea0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/1274422129 shutdown_connections 2024-03-07T07:19:27.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b4070400 0x7fe6b40708b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/1274422129 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/1274422129 >> 172.21.15.64:0/1274422129 conn(0x7fe6b4072490 msgr2=0x7fe6b4072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/1274422129 shutdown_connections 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.626+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/1274422129 wait complete. 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 Processor -- start 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 -- start start 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b4070400 0x7fe6b413ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b4144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b415e750 0x7fe6b40cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4167d50 con 0x7fe6b4070400 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4070df0 con 0x7fe6b415e750 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b8bc9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4167ed0 con 0x7fe6b40aec30 2024-03-07T07:19:27.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b4070400 0x7fe6b413ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b4070400 0x7fe6b413ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34624/0 (socket says 172.21.15.64:34624) 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b259c640 1 -- 172.21.15.64:0/276400974 learned_addr learned my addr 172.21.15.64:0/276400974 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b2d9d640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b415e750 0x7fe6b40cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.628+0000 7fe6b1d9b640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b4144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b259c640 1 -- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 msgr2=0x7fe6b4144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b259c640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b4144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b259c640 1 -- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b415e750 msgr2=0x7fe6b40cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b259c640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b415e750 0x7fe6b40cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b259c640 1 -- 172.21.15.64:0/276400974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4159a90 con 0x7fe6b4070400 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b2d9d640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b415e750 0x7fe6b40cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b259c640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b4070400 0x7fe6b413ba70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00c3c0 tx=0x7fe6ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6aaffd640 1 -- 172.21.15.64:0/276400974 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac017070 con 0x7fe6b4070400 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6aaffd640 1 -- 172.21.15.64:0/276400974 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac005ce0 con 0x7fe6b4070400 2024-03-07T07:19:27.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6aaffd640 1 -- 172.21.15.64:0/276400974 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0072f0 con 0x7fe6b4070400 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b4159d70 con 0x7fe6b4070400 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.629+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b415d620 con 0x7fe6b4070400 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.630+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b4062c90 con 0x7fe6b4070400 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.630+0000 7fe6aaffd640 1 -- 172.21.15.64:0/276400974 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe6ac01a070 con 0x7fe6b4070400 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.631+0000 7fe6aaffd640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.631+0000 7fe6aaffd640 1 -- 172.21.15.64:0/276400974 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6ac0894f0 con 0x7fe6b4070400 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.631+0000 7fe6b1d9b640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.631+0000 7fe6b1d9b640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4071f50 tx=0x7fe69c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.635+0000 7fe6aaffd640 1 -- 172.21.15.64:0/276400974 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe6ac057d30 con 0x7fe6b4070400 2024-03-07T07:19:27.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.818+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6b40ccf90 con 0x7fe6b4070400 2024-03-07T07:19:27.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.819+0000 7fe6aaffd640 1 -- 172.21.15.64:0/276400974 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fe6ac05bdd0 con 0x7fe6b4070400 2024-03-07T07:19:27.820 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:27.820 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69406b4d0 msgr2=0x7fe69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4071f50 tx=0x7fe69c009190 comp rx=0 tx=0).stop 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b4070400 msgr2=0x7fe6b413ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b4070400 0x7fe6b413ba70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00c3c0 tx=0x7fe6ac00c880 comp rx=0 tx=0).stop 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 shutdown_connections 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b415e750 0x7fe6b40cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b40aec30 0x7fe6b4144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 --2- 172.21.15.64:0/276400974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b4070400 0x7fe6b413ba70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 >> 172.21.15.64:0/276400974 conn(0x7fe6b4072490 msgr2=0x7fe6b415eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 shutdown_connections 2024-03-07T07:19:27.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.821+0000 7fe6b8bc9640 1 -- 172.21.15.64:0/276400974 wait complete. 2024-03-07T07:19:27.832 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:27.833 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:27.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.936+0000 7fa2a7d72640 1 Processor -- start 2024-03-07T07:19:27.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.936+0000 7fa2a7d72640 1 -- start start 2024-03-07T07:19:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a7d72640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa2a015e350 0x7fa2a0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a7d72640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2a0071990 0x7fa2a01623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a7d72640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a0071ff0 con 0x7fa2a00b7200 2024-03-07T07:19:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a7d72640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a00d2160 con 0x7fa2a0071990 2024-03-07T07:19:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a7d72640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a00d23b0 con 0x7fa2a015e350 2024-03-07T07:19:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a630f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2a0071990 0x7fa2a01623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a630f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2a0071990 0x7fa2a01623f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34626/0 (socket says 172.21.15.64:34626) 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a630f640 1 -- 172.21.15.64:0/3229691402 learned_addr learned my addr 172.21.15.64:0/3229691402 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a5b0e640 1 --2- 172.21.15.64:0/3229691402 >> v2:172.21.15.120:3300/0 conn(0x7fa2a015e350 0x7fa2a0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a630f640 1 -- 172.21.15.64:0/3229691402 >> v2:172.21.15.120:3300/0 conn(0x7fa2a015e350 msgr2=0x7fa2a0161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a630f640 1 --2- 172.21.15.64:0/3229691402 >> v2:172.21.15.120:3300/0 conn(0x7fa2a015e350 0x7fa2a0161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a630f640 1 -- 172.21.15.64:0/3229691402 >> v1:172.21.15.120:6789/0 conn(0x7fa2a00b7200 legacy=0x7fa2a0071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.937+0000 7fa2a630f640 1 -- 172.21.15.64:0/3229691402 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00d2690 con 0x7fa2a0071990 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.938+0000 7fa2a630f640 1 --2- 172.21.15.64:0/3229691402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0071990 0x7fa2a01623f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00b6b90 tx=0x7fa29802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4610b7b74ed90e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.938+0000 7fa2a4b0c640 1 -- 172.21.15.64:0/3229691402 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29802cb20 con 0x7fa2a0071990 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.938+0000 7fa2a4b0c640 1 -- 172.21.15.64:0/3229691402 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29802ccc0 con 0x7fa2a0071990 2024-03-07T07:19:27.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.938+0000 7fa2a4b0c640 1 -- 172.21.15.64:0/3229691402 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298034610 con 0x7fa2a0071990 2024-03-07T07:19:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 -- 172.21.15.64:0/3229691402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0071990 msgr2=0x7fa2a01623f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/3229691402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0071990 0x7fa2a01623f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00b6b90 tx=0x7fa29802bf80 comp rx=0 tx=0).stop 2024-03-07T07:19:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 -- 172.21.15.64:0/3229691402 shutdown_connections 2024-03-07T07:19:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/3229691402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0071990 0x7fa2a01623f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/3229691402 >> v2:172.21.15.120:3300/0 conn(0x7fa2a015e350 0x7fa2a0161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 -- 172.21.15.64:0/3229691402 >> 172.21.15.64:0/3229691402 conn(0x7fa2a00d0e70 msgr2=0x7fa2a00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 -- 172.21.15.64:0/3229691402 shutdown_connections 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.939+0000 7fa2a7d72640 1 -- 172.21.15.64:0/3229691402 wait complete. 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.940+0000 7fa2a7d72640 1 Processor -- start 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.940+0000 7fa2a7d72640 1 -- start start 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2a0071990 0x7fa2a01551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2a00b7200 0x7fa2a015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a015e350 0x7fa2a015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00d2e90 con 0x7fa2a015e350 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a015e730 con 0x7fa2a0071990 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00d3010 con 0x7fa2a00b7200 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a015e350 0x7fa2a015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a015e350 0x7fa2a015cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34640/0 (socket says 172.21.15.64:34640) 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 -- 172.21.15.64:0/663687179 learned_addr learned my addr 172.21.15.64:0/663687179 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a5b0e640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2a0071990 0x7fa2a01551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a530d640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2a00b7200 0x7fa2a015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 -- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2a00b7200 msgr2=0x7fa2a015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2a00b7200 0x7fa2a015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 -- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2a0071990 msgr2=0x7fa2a01551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2a0071990 0x7fa2a01551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 -- 172.21.15.64:0/663687179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0160de0 con 0x7fa2a015e350 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a630f640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a015e350 0x7fa2a015cd10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa298036040 tx=0x7fa298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2967fc640 1 -- 172.21.15.64:0/663687179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298043020 con 0x7fa2a015e350 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2967fc640 1 -- 172.21.15.64:0/663687179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29803e450 con 0x7fa2a015e350 2024-03-07T07:19:27.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2967fc640 1 -- 172.21.15.64:0/663687179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298009510 con 0x7fa2a015e350 2024-03-07T07:19:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a01610c0 con 0x7fa2a015e350 2024-03-07T07:19:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.941+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a015e950 con 0x7fa2a015e350 2024-03-07T07:19:27.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.942+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a015eb50 con 0x7fa2a015e350 2024-03-07T07:19:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.943+0000 7fa2967fc640 1 -- 172.21.15.64:0/663687179 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa298046020 con 0x7fa2a015e350 2024-03-07T07:19:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.943+0000 7fa2967fc640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa28406b4d0 0x7fa28406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.943+0000 7fa2967fc640 1 -- 172.21.15.64:0/663687179 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2980ac920 con 0x7fa2a015e350 2024-03-07T07:19:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.943+0000 7fa2a5b0e640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa28406b4d0 0x7fa28406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:27.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.944+0000 7fa2a5b0e640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa28406b4d0 0x7fa28406d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009aa0 tx=0x7fa29c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:27.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:27.947+0000 7fa2967fc640 1 -- 172.21.15.64:0/663687179 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa2a015eb50 con 0x7fa2a015e350 2024-03-07T07:19:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.126+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2a0062c90 con 0x7fa2a015e350 2024-03-07T07:19:28.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.126+0000 7fa2967fc640 1 -- 172.21.15.64:0/663687179 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa29804d030 con 0x7fa2a015e350 2024-03-07T07:19:28.126 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa28406b4d0 msgr2=0x7fa28406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa28406b4d0 0x7fa28406d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009aa0 tx=0x7fa29c009190 comp rx=0 tx=0).stop 2024-03-07T07:19:28.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a015e350 msgr2=0x7fa2a015cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a015e350 0x7fa2a015cd10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa298036040 tx=0x7fa298004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 shutdown_connections 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa28406b4d0 0x7fa28406d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a015e350 0x7fa2a015cd10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2a00b7200 0x7fa2a015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 --2- 172.21.15.64:0/663687179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2a0071990 0x7fa2a01551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 >> 172.21.15.64:0/663687179 conn(0x7fa2a00d0e70 msgr2=0x7fa2a00b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 shutdown_connections 2024-03-07T07:19:28.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.128+0000 7fa2a7d72640 1 -- 172.21.15.64:0/663687179 wait complete. 2024-03-07T07:19:28.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:28.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.243+0000 7f1ec171d640 1 Processor -- start 2024-03-07T07:19:28.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.243+0000 7f1ec171d640 1 -- start start 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ec171d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ebc162110 0x7f1ebc1624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ec171d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1ebc0d3700 0x7f1ebc15efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ec171d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc162a30 con 0x7f1ebc070bc0 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ec171d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc071610 con 0x7f1ebc162110 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ec171d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0718f0 con 0x7f1ebc0d3700 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ebaffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1ebc162110 0x7f1ebc1624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1eb27fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1ebc070bc0 0x7f1ebc15e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48744/0 (socket says 172.21.15.64:48744) 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1eb27fc640 1 -- 172.21.15.64:0/4182220228 learned_addr learned my addr 172.21.15.64:0/4182220228 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ebb7fe640 1 --2- 172.21.15.64:0/4182220228 >> v2:172.21.15.120:3300/0 conn(0x7f1ebc0d3700 0x7f1ebc15efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ebaffd640 1 -- 172.21.15.64:0/4182220228 >> v2:172.21.15.120:3300/0 conn(0x7f1ebc0d3700 msgr2=0x7f1ebc15efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ebaffd640 1 --2- 172.21.15.64:0/4182220228 >> v2:172.21.15.120:3300/0 conn(0x7f1ebc0d3700 0x7f1ebc15efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ebaffd640 1 -- 172.21.15.64:0/4182220228 >> v1:172.21.15.64:6789/0 conn(0x7f1ebc070bc0 legacy=0x7f1ebc15e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1ebaffd640 1 -- 172.21.15.64:0/4182220228 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc071bd0 con 0x7f1ebc162110 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.244+0000 7f1eba7fc640 1 -- 172.21.15.64:0/4182220228 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735733931 0 0) 0x7f1ebc162a30 con 0x7f1ebc070bc0 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1ebaffd640 1 --2- 172.21.15.64:0/4182220228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc162110 0x7f1ebc1624f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc15fcb0 tx=0x7f1eac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8566d29f2d829357 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1eba7fc640 1 -- 172.21.15.64:0/4182220228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac035020 con 0x7f1ebc162110 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1eba7fc640 1 -- 172.21.15.64:0/4182220228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac031830 con 0x7f1ebc162110 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1eba7fc640 1 -- 172.21.15.64:0/4182220228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac031c00 con 0x7f1ebc162110 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1ec171d640 1 -- 172.21.15.64:0/4182220228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc162110 msgr2=0x7f1ebc1624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1ec171d640 1 --2- 172.21.15.64:0/4182220228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc162110 0x7f1ebc1624f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc15fcb0 tx=0x7f1eac02b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1ec171d640 1 -- 172.21.15.64:0/4182220228 shutdown_connections 2024-03-07T07:19:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1ec171d640 1 --2- 172.21.15.64:0/4182220228 >> v2:172.21.15.120:3300/0 conn(0x7f1ebc0d3700 0x7f1ebc15efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1ec171d640 1 --2- 172.21.15.64:0/4182220228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc162110 0x7f1ebc1624f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.245+0000 7f1ec171d640 1 -- 172.21.15.64:0/4182220228 >> 172.21.15.64:0/4182220228 conn(0x7f1ebc0d0e10 msgr2=0x7f1ebc0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.246+0000 7f1ec171d640 1 -- 172.21.15.64:0/4182220228 shutdown_connections 2024-03-07T07:19:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.246+0000 7f1ec171d640 1 -- 172.21.15.64:0/4182220228 wait complete. 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 Processor -- start 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 -- start start 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ebc070bc0 0x7f1ebc15d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0d3700 0x7f1ebc0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc162110 0x7f1ebc0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc162e00 con 0x7f1ebc0d3700 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0d21d0 con 0x7f1ebc162110 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.247+0000 7f1ec171d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc162f80 con 0x7f1ebc070bc0 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0d3700 0x7f1ebc0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0d3700 0x7f1ebc0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34650/0 (socket says 172.21.15.64:34650) 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 -- 172.21.15.64:0/1244011669 learned_addr learned my addr 172.21.15.64:0/1244011669 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1ebb7fe640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc162110 0x7f1ebc0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1ebaffd640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ebc070bc0 0x7f1ebc15d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 -- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ebc070bc0 msgr2=0x7f1ebc15d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ebc070bc0 0x7f1ebc15d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 -- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc162110 msgr2=0x7f1ebc0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc162110 0x7f1ebc0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 -- 172.21.15.64:0/1244011669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0abe40 con 0x7f1ebc0d3700 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb27fc640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0d3700 0x7f1ebc0acee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c4d0 tx=0x7f1ea400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb3fff640 1 -- 172.21.15.64:0/1244011669 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4018070 con 0x7f1ebc0d3700 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb3fff640 1 -- 172.21.15.64:0/1244011669 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4007500 con 0x7f1ebc0d3700 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc0ac120 con 0x7f1ebc0d3700 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1eb3fff640 1 -- 172.21.15.64:0/1244011669 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004a20 con 0x7f1ebc0d3700 2024-03-07T07:19:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.248+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc0ac6e0 con 0x7f1ebc0d3700 2024-03-07T07:19:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.249+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc062c90 con 0x7f1ebc0d3700 2024-03-07T07:19:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.250+0000 7f1eb3fff640 1 -- 172.21.15.64:0/1244011669 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1ea401b070 con 0x7f1ebc0d3700 2024-03-07T07:19:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.250+0000 7f1eb3fff640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e9c06b4d0 0x7f1e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.250+0000 7f1eb3fff640 1 -- 172.21.15.64:0/1244011669 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ea40894a0 con 0x7f1ebc0d3700 2024-03-07T07:19:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.250+0000 7f1ebaffd640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e9c06b4d0 0x7f1e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.251+0000 7f1ebaffd640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e9c06b4d0 0x7f1e9c06d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1eac036000 tx=0x7f1eac03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.254+0000 7f1eb3fff640 1 -- 172.21.15.64:0/1244011669 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1ea4057cd0 con 0x7f1ebc0d3700 2024-03-07T07:19:28.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.437+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ebc0706d0 con 0x7f1ebc0d3700 2024-03-07T07:19:28.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.437+0000 7f1eb3fff640 1 -- 172.21.15.64:0/1244011669 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ea405bd70 con 0x7f1ebc0d3700 2024-03-07T07:19:28.438 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:28.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e9c06b4d0 msgr2=0x7f1e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e9c06b4d0 0x7f1e9c06d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1eac036000 tx=0x7f1eac03c040 comp rx=0 tx=0).stop 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0d3700 msgr2=0x7f1ebc0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0d3700 0x7f1ebc0acee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c4d0 tx=0x7f1ea400c990 comp rx=0 tx=0).stop 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 shutdown_connections 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1e9c06b4d0 0x7f1e9c06d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0d3700 0x7f1ebc0acee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ebc162110 0x7f1ebc0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 --2- 172.21.15.64:0/1244011669 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ebc070bc0 0x7f1ebc15d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 >> 172.21.15.64:0/1244011669 conn(0x7f1ebc0d0e10 msgr2=0x7f1ebc0b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 shutdown_connections 2024-03-07T07:19:28.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.439+0000 7f1ec171d640 1 -- 172.21.15.64:0/1244011669 wait complete. 2024-03-07T07:19:28.450 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:28.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aaaa1e640 1 Processor -- start 2024-03-07T07:19:28.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aaaa1e640 1 -- start start 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aaaa1e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1aa415b370 0x7f1aa40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aaaa1e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa406d740 con 0x7f1aa406d180 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aaaa1e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa406da20 con 0x7f1aa40b3ed0 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aaaa1e640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa406dd00 con 0x7f1aa415b370 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aa3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1aa415b370 0x7f1aa40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aa3fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1aa415b370 0x7f1aa40cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34664/0 (socket says 172.21.15.64:34664) 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aa37fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1aa406d180 0x7f1aa415b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48756/0 (socket says 172.21.15.64:48756) 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.554+0000 7f1aa3fff640 1 -- 172.21.15.64:0/1601213130 learned_addr learned my addr 172.21.15.64:0/1601213130 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.555+0000 7f1aa3fff640 1 -- 172.21.15.64:0/1601213130 >> v1:172.21.15.120:6789/0 conn(0x7f1aa40b3ed0 legacy=0x7f1aa4164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.555+0000 7f1aa3fff640 1 -- 172.21.15.64:0/1601213130 >> v1:172.21.15.64:6789/0 conn(0x7f1aa406d180 legacy=0x7f1aa415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.555+0000 7f1aa3fff640 1 -- 172.21.15.64:0/1601213130 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa406dfe0 con 0x7f1aa415b370 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.555+0000 7f1aa3fff640 1 --2- 172.21.15.64:0/1601213130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa415b370 0x7f1aa40cef50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009870 tx=0x7f1a9802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738eec750a51f799 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aa2ffd640 1 -- 172.21.15.64:0/1601213130 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9802f840 con 0x7f1aa415b370 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aa2ffd640 1 -- 172.21.15.64:0/1601213130 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9802f9e0 con 0x7f1aa415b370 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aa2ffd640 1 -- 172.21.15.64:0/1601213130 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9802fdb0 con 0x7f1aa415b370 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/1601213130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa415b370 msgr2=0x7f1aa40cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/1601213130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa415b370 0x7f1aa40cef50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009870 tx=0x7f1a9802ba20 comp rx=0 tx=0).stop 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/1601213130 shutdown_connections 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/1601213130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa415b370 0x7f1aa40cef50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/1601213130 >> 172.21.15.64:0/1601213130 conn(0x7f1aa40b1d30 msgr2=0x7f1aa40b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:28.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/1601213130 shutdown_connections 2024-03-07T07:19:28.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.556+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/1601213130 wait complete. 2024-03-07T07:19:28.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.557+0000 7f1aaaa1e640 1 Processor -- start 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aaaa1e640 1 -- start start 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aaaa1e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa406d180 0x7f1aa41f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aaaa1e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1aa40b3ed0 0x7f1aa414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aaaa1e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa415b370 0x7f1aa4152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aaaa1e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415bfe0 con 0x7f1aa406d180 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aaaa1e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415be60 con 0x7f1aa415b370 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aaaa1e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415c160 con 0x7f1aa40b3ed0 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aa3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa406d180 0x7f1aa41f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aa3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa406d180 0x7f1aa41f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34680/0 (socket says 172.21.15.64:34680) 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aa3fff640 1 -- 172.21.15.64:0/897848127 learned_addr learned my addr 172.21.15.64:0/897848127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aa8fbb640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa415b370 0x7f1aa4152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.558+0000 7f1aa37fe640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1aa40b3ed0 0x7f1aa414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa3fff640 1 -- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1aa40b3ed0 msgr2=0x7f1aa414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa3fff640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1aa40b3ed0 0x7f1aa414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa3fff640 1 -- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa415b370 msgr2=0x7f1aa4152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa3fff640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa415b370 0x7f1aa4152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa3fff640 1 -- 172.21.15.64:0/897848127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4152d00 con 0x7f1aa406d180 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa3fff640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa406d180 0x7f1aa41f7c20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a98000c00 tx=0x7f1a98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa0ff9640 1 -- 172.21.15.64:0/897848127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98041400 con 0x7f1aa406d180 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa0ff9640 1 -- 172.21.15.64:0/897848127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a98046070 con 0x7f1aa406d180 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aa0ff9640 1 -- 172.21.15.64:0/897848127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98041800 con 0x7f1aa406d180 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa4152fe0 con 0x7f1aa406d180 2024-03-07T07:19:28.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.559+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa4146f10 con 0x7f1aa406d180 2024-03-07T07:19:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.560+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa4062c90 con 0x7f1aa406d180 2024-03-07T07:19:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.560+0000 7f1aa0ff9640 1 -- 172.21.15.64:0/897848127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1a98040070 con 0x7f1aa406d180 2024-03-07T07:19:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.561+0000 7f1aa0ff9640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1a9006b4d0 0x7f1a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.561+0000 7f1aa0ff9640 1 -- 172.21.15.64:0/897848127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a980ac850 con 0x7f1aa406d180 2024-03-07T07:19:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.561+0000 7f1aa37fe640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1a9006b4d0 0x7f1a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.561+0000 7f1aa37fe640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1a9006b4d0 0x7f1a9006d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4073160 tx=0x7f1a94000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.565+0000 7f1aa0ff9640 1 -- 172.21.15.64:0/897848127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1a98033030 con 0x7f1aa406d180 2024-03-07T07:19:28.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.745+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1aa4072eb0 con 0x7f1aa406d180 2024-03-07T07:19:28.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.745+0000 7f1aa0ff9640 1 -- 172.21.15.64:0/897848127 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a9807b110 con 0x7f1aa406d180 2024-03-07T07:19:28.745 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1a9006b4d0 msgr2=0x7f1a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1a9006b4d0 0x7f1a9006d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4073160 tx=0x7f1a94000ed0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa406d180 msgr2=0x7f1aa41f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa406d180 0x7f1aa41f7c20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a98000c00 tx=0x7f1a98004730 comp rx=0 tx=0).stop 2024-03-07T07:19:28.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 shutdown_connections 2024-03-07T07:19:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1a9006b4d0 0x7f1a9006d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1aa415b370 0x7f1aa4152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1aa40b3ed0 0x7f1aa414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 --2- 172.21.15.64:0/897848127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa406d180 0x7f1aa41f7c20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 >> 172.21.15.64:0/897848127 conn(0x7f1aa40b1d30 msgr2=0x7f1aa40cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 shutdown_connections 2024-03-07T07:19:28.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.747+0000 7f1aaaa1e640 1 -- 172.21.15.64:0/897848127 wait complete. 2024-03-07T07:19:28.758 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:28.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.858+0000 7f98924b6640 1 Processor -- start 2024-03-07T07:19:28.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f98924b6640 1 -- start start 2024-03-07T07:19:28.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f98924b6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f988c145350 0x7f988c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f98924b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f98924b6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f988c070980 0x7f988c06eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f98924b6640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c0cf1f0 con 0x7f988c145350 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f98924b6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c070e10 con 0x7f988c070980 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f98924b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c15ffc0 con 0x7f988c15c580 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f988bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f988c145350 0x7f988c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f988bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f988c145350 0x7f988c0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34682/0 (socket says 172.21.15.64:34682) 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f988bfff640 1 -- 172.21.15.64:0/3911218363 learned_addr learned my addr 172.21.15.64:0/3911218363 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f9890a53640 1 --2- 172.21.15.64:0/3911218363 >> v2:172.21.15.120:3300/0 conn(0x7f988c070980 0x7f988c06eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.859+0000 7f988b7fe640 1 --2- 172.21.15.64:0/3911218363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.860+0000 7f988bfff640 1 -- 172.21.15.64:0/3911218363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 msgr2=0x7f988c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.860+0000 7f988bfff640 1 --2- 172.21.15.64:0/3911218363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.860+0000 7f988bfff640 1 -- 172.21.15.64:0/3911218363 >> v2:172.21.15.120:3300/0 conn(0x7f988c070980 msgr2=0x7f988c06eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.860+0000 7f988bfff640 1 --2- 172.21.15.64:0/3911218363 >> v2:172.21.15.120:3300/0 conn(0x7f988c070980 0x7f988c06eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.860+0000 7f988bfff640 1 -- 172.21.15.64:0/3911218363 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c1602a0 con 0x7f988c145350 2024-03-07T07:19:28.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.860+0000 7f9890a53640 1 --2- 172.21.15.64:0/3911218363 >> v2:172.21.15.120:3300/0 conn(0x7f988c070980 0x7f988c06eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.860+0000 7f988bfff640 1 --2- 172.21.15.64:0/3911218363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c0cecb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f988c0b3f30 tx=0x7f988002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8ec69946e6b543 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f988affd640 1 -- 172.21.15.64:0/3911218363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880035020 con 0x7f988c145350 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f988affd640 1 -- 172.21.15.64:0/3911218363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988002d040 con 0x7f988c145350 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f988affd640 1 -- 172.21.15.64:0/3911218363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880031a80 con 0x7f988c145350 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 -- 172.21.15.64:0/3911218363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 msgr2=0x7f988c0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 --2- 172.21.15.64:0/3911218363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c0cecb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f988c0b3f30 tx=0x7f988002b900 comp rx=0 tx=0).stop 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 -- 172.21.15.64:0/3911218363 shutdown_connections 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 --2- 172.21.15.64:0/3911218363 >> v2:172.21.15.120:3300/0 conn(0x7f988c070980 0x7f988c06eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 --2- 172.21.15.64:0/3911218363 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 --2- 172.21.15.64:0/3911218363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c0cecb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 -- 172.21.15.64:0/3911218363 >> 172.21.15.64:0/3911218363 conn(0x7f988c06b040 msgr2=0x7f988c06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 -- 172.21.15.64:0/3911218363 shutdown_connections 2024-03-07T07:19:28.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.861+0000 7f98924b6640 1 -- 172.21.15.64:0/3911218363 wait complete. 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.862+0000 7f98924b6640 1 Processor -- start 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.862+0000 7f98924b6640 1 -- start start 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f98924b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f988c070980 0x7f988c13a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f98924b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c13f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f98924b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c13fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f98924b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c145ac0 con 0x7f988c145350 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f98924b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c160860 con 0x7f988c070980 2024-03-07T07:19:28.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f98924b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c15c9e0 con 0x7f988c15c580 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f988b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c13f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f988b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c13f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34684/0 (socket says 172.21.15.64:34684) 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f988b7fe640 1 -- 172.21.15.64:0/86302875 learned_addr learned my addr 172.21.15.64:0/86302875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f988bfff640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f988c070980 0x7f988c13a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.863+0000 7f9890a53640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c13fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f988b7fe640 1 -- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 msgr2=0x7f988c13fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f988b7fe640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c13fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f988b7fe640 1 -- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f988c070980 msgr2=0x7f988c13a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f988b7fe640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f988c070980 0x7f988c13a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f988b7fe640 1 -- 172.21.15.64:0/86302875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c13b2d0 con 0x7f988c145350 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f988b7fe640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c13f9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e870 tx=0x7f987c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f9888ff9640 1 -- 172.21.15.64:0/86302875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c017070 con 0x7f988c145350 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f9888ff9640 1 -- 172.21.15.64:0/86302875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c00ee60 con 0x7f988c145350 2024-03-07T07:19:28.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c13b5b0 con 0x7f988c145350 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f9888ff9640 1 -- 172.21.15.64:0/86302875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00cb80 con 0x7f988c145350 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.864+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c13ae50 con 0x7f988c145350 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.865+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c062c90 con 0x7f988c145350 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.865+0000 7f9888ff9640 1 -- 172.21.15.64:0/86302875 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f987c01a070 con 0x7f988c145350 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.866+0000 7f9888ff9640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f987806b4d0 0x7f987806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.866+0000 7f988bfff640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f987806b4d0 0x7f987806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.866+0000 7f9888ff9640 1 -- 172.21.15.64:0/86302875 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f987c08a300 con 0x7f988c145350 2024-03-07T07:19:28.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.866+0000 7f988bfff640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f987806b4d0 0x7f987806d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9880036000 tx=0x7f988003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:28.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:28.870+0000 7f9888ff9640 1 -- 172.21.15.64:0/86302875 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f987c058b30 con 0x7f988c145350 2024-03-07T07:19:29.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.088+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f988c06f440 con 0x7f988c145350 2024-03-07T07:19:29.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.089+0000 7f9888ff9640 1 -- 172.21.15.64:0/86302875 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7f987c05cbd0 con 0x7f988c145350 2024-03-07T07:19:29.089 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:29.089 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:29.089 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.090+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f987806b4d0 msgr2=0x7f987806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.090+0000 7f98924b6640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f987806b4d0 0x7f987806d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9880036000 tx=0x7f988003c040 comp rx=0 tx=0).stop 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.090+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 msgr2=0x7f988c13f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.090+0000 7f98924b6640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c13f9a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e870 tx=0x7f987c00ec70 comp rx=0 tx=0).stop 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 shutdown_connections 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f987806b4d0 0x7f987806d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f988c145350 0x7f988c13f9a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f988c15c580 0x7f988c13fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 --2- 172.21.15.64:0/86302875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f988c070980 0x7f988c13a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 >> 172.21.15.64:0/86302875 conn(0x7f988c06b040 msgr2=0x7f988c06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:29.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 shutdown_connections 2024-03-07T07:19:29.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.091+0000 7f98924b6640 1 -- 172.21.15.64:0/86302875 wait complete. 2024-03-07T07:19:29.102 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.203+0000 7ff5a7178640 1 Processor -- start 2024-03-07T07:19:29.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.203+0000 7ff5a7178640 1 -- start start 2024-03-07T07:19:29.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a7178640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff5a01416d0 0x7ff5a015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a7178640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00d3490 0x7ff5a00d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a7178640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a0141c60 con 0x7ff5a0162030 2024-03-07T07:19:29.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a7178640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a00d3e40 con 0x7ff5a01416d0 2024-03-07T07:19:29.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a7178640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00b44b0 con 0x7ff5a00d3490 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a5715640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00d3490 0x7ff5a00d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff59ffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff5a01416d0 0x7ff5a015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a4f14640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff5a0162030 0x7ff5a0162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:47874/0 (socket says 172.21.15.64:47874) 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a4f14640 1 -- 172.21.15.64:0/110288614 learned_addr learned my addr 172.21.15.64:0/110288614 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff59ffff640 1 -- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00d3490 msgr2=0x7ff5a00d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff59ffff640 1 --2- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00d3490 0x7ff5a00d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff59ffff640 1 -- 172.21.15.64:0/110288614 >> v1:172.21.15.120:6789/0 conn(0x7ff5a0162030 legacy=0x7ff5a0162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff59ffff640 1 -- 172.21.15.64:0/110288614 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a015ecd0 con 0x7ff5a01416d0 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff59f7fe640 1 -- 172.21.15.64:0/110288614 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065020002 0 0) 0x7ff5a0141c60 con 0x7ff5a0162030 2024-03-07T07:19:29.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.204+0000 7ff5a5715640 1 --2- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00d3490 0x7ff5a00d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.205+0000 7ff59ffff640 1 --2- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a01416d0 0x7ff5a015e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0097a0 tx=0x7ff58c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1495352b4d56f7d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.205+0000 7ff59f7fe640 1 -- 172.21.15.64:0/110288614 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c035020 con 0x7ff5a01416d0 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.205+0000 7ff59f7fe640 1 -- 172.21.15.64:0/110288614 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c031830 con 0x7ff5a01416d0 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.205+0000 7ff59f7fe640 1 -- 172.21.15.64:0/110288614 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c031c00 con 0x7ff5a01416d0 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.205+0000 7ff5a7178640 1 -- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a01416d0 msgr2=0x7ff5a015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.205+0000 7ff5a7178640 1 --2- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a01416d0 0x7ff5a015e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0097a0 tx=0x7ff58c02b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.206+0000 7ff5a7178640 1 -- 172.21.15.64:0/110288614 shutdown_connections 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.206+0000 7ff5a7178640 1 --2- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00d3490 0x7ff5a00d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.206+0000 7ff5a7178640 1 --2- 172.21.15.64:0/110288614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a01416d0 0x7ff5a015e680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.206+0000 7ff5a7178640 1 -- 172.21.15.64:0/110288614 >> 172.21.15.64:0/110288614 conn(0x7ff5a0070400 msgr2=0x7ff5a00d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.206+0000 7ff5a7178640 1 -- 172.21.15.64:0/110288614 shutdown_connections 2024-03-07T07:19:29.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.206+0000 7ff5a7178640 1 -- 172.21.15.64:0/110288614 wait complete. 2024-03-07T07:19:29.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.207+0000 7ff5a7178640 1 Processor -- start 2024-03-07T07:19:29.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.207+0000 7ff5a7178640 1 -- start start 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.207+0000 7ff5a7178640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a00d3490 0x7ff5a0158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a7178640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a01416d0 0x7ff5a01457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a7178640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a0162030 0x7ff5a015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a7178640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a0162b20 con 0x7ff5a01416d0 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a7178640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a015f290 con 0x7ff5a00d3490 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a7178640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a015f570 con 0x7ff5a0162030 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a4f14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a00d3490 0x7ff5a0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a01416d0 0x7ff5a01457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a01416d0 0x7ff5a01457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34694/0 (socket says 172.21.15.64:34694) 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a4f14640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a00d3490 0x7ff5a0158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:34528/0 (socket says 172.21.15.64:34528) 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 -- 172.21.15.64:0/4209730098 learned_addr learned my addr 172.21.15.64:0/4209730098 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a5715640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a0162030 0x7ff5a015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 -- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a0162030 msgr2=0x7ff5a015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a0162030 0x7ff5a015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 -- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a00d3490 msgr2=0x7ff5a0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a00d3490 0x7ff5a0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 -- 172.21.15.64:0/4209730098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a01461f0 con 0x7ff5a01416d0 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59ffff640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a01416d0 0x7ff5a01457a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff58c000c00 tx=0x7ff58c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59d7fa640 1 -- 172.21.15.64:0/4209730098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c035070 con 0x7ff5a01416d0 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59d7fa640 1 -- 172.21.15.64:0/4209730098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c004230 con 0x7ff5a01416d0 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a01464d0 con 0x7ff5a01416d0 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff59d7fa640 1 -- 172.21.15.64:0/4209730098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c03a630 con 0x7ff5a01416d0 2024-03-07T07:19:29.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.208+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a0146a90 con 0x7ff5a01416d0 2024-03-07T07:19:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.209+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a0062c90 con 0x7ff5a01416d0 2024-03-07T07:19:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.210+0000 7ff59d7fa640 1 -- 172.21.15.64:0/4209730098 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff58c038020 con 0x7ff5a01416d0 2024-03-07T07:19:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.210+0000 7ff59d7fa640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff58406b4d0 0x7ff58406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.210+0000 7ff59d7fa640 1 -- 172.21.15.64:0/4209730098 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff58c0abce0 con 0x7ff5a01416d0 2024-03-07T07:19:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.210+0000 7ff5a4f14640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff58406b4d0 0x7ff58406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.211+0000 7ff5a4f14640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff58406b4d0 0x7ff58406d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff594000f60 tx=0x7ff594007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.214+0000 7ff59d7fa640 1 -- 172.21.15.64:0/4209730098 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff58c040070 con 0x7ff5a01416d0 2024-03-07T07:19:29.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.399+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5a0071950 con 0x7ff5a01416d0 2024-03-07T07:19:29.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.400+0000 7ff59d7fa640 1 -- 172.21.15.64:0/4209730098 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7ff58c07ab60 con 0x7ff5a01416d0 2024-03-07T07:19:29.401 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:29.401 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff58406b4d0 msgr2=0x7ff58406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff58406b4d0 0x7ff58406d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff594000f60 tx=0x7ff594007d50 comp rx=0 tx=0).stop 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a01416d0 msgr2=0x7ff5a01457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a01416d0 0x7ff5a01457a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff58c000c00 tx=0x7ff58c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 shutdown_connections 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff58406b4d0 0x7ff58406d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a01416d0 0x7ff5a01457a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a0162030 0x7ff5a015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 --2- 172.21.15.64:0/4209730098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5a00d3490 0x7ff5a0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 >> 172.21.15.64:0/4209730098 conn(0x7ff5a0070400 msgr2=0x7ff5a00d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 shutdown_connections 2024-03-07T07:19:29.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.403+0000 7ff5a7178640 1 -- 172.21.15.64:0/4209730098 wait complete. 2024-03-07T07:19:29.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:29.414 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.518+0000 7f0c8ab1d640 1 Processor -- start 2024-03-07T07:19:29.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.518+0000 7f0c8ab1d640 1 -- start start 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.518+0000 7f0c8ab1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c8415e870 0x7f0c840cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c8ab1d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0c840d3310 0x7f0c84167e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c8ab1d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c84070eb0 con 0x7f0c840cea70 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c8ab1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c84071190 con 0x7f0c8415e870 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c8ab1d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c84071470 con 0x7f0c840d3310 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c888b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c8415e870 0x7f0c840cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c888b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0c8415e870 0x7f0c840cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34706/0 (socket says 172.21.15.64:34706) 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c888b9640 1 -- 172.21.15.64:0/112104923 learned_addr learned my addr 172.21.15.64:0/112104923 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c890ba640 1 --2- 172.21.15.64:0/112104923 >> v2:172.21.15.120:3300/0 conn(0x7f0c840d3310 0x7f0c84167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c888b9640 1 -- 172.21.15.64:0/112104923 >> v2:172.21.15.120:3300/0 conn(0x7f0c840d3310 msgr2=0x7f0c84167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c888b9640 1 --2- 172.21.15.64:0/112104923 >> v2:172.21.15.120:3300/0 conn(0x7f0c840d3310 0x7f0c84167e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c888b9640 1 -- 172.21.15.64:0/112104923 >> v1:172.21.15.120:6789/0 conn(0x7f0c840cea70 legacy=0x7f0c8415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.519+0000 7f0c888b9640 1 -- 172.21.15.64:0/112104923 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c84071750 con 0x7f0c8415e870 2024-03-07T07:19:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c888b9640 1 --2- 172.21.15.64:0/112104923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c8415e870 0x7f0c840cf190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c840cfae0 tx=0x7f0c7802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df12ae490a9f590 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c837fe640 1 -- 172.21.15.64:0/112104923 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78034070 con 0x7f0c8415e870 2024-03-07T07:19:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c837fe640 1 -- 172.21.15.64:0/112104923 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c78030860 con 0x7f0c8415e870 2024-03-07T07:19:29.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c837fe640 1 -- 172.21.15.64:0/112104923 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78030c30 con 0x7f0c8415e870 2024-03-07T07:19:29.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/112104923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c8415e870 msgr2=0x7f0c840cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/112104923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c8415e870 0x7f0c840cf190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c840cfae0 tx=0x7f0c7802b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/112104923 shutdown_connections 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/112104923 >> v2:172.21.15.120:3300/0 conn(0x7f0c840d3310 0x7f0c84167e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/112104923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c8415e870 0x7f0c840cf190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/112104923 >> 172.21.15.64:0/112104923 conn(0x7f0c840d11e0 msgr2=0x7f0c840d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/112104923 shutdown_connections 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.520+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/112104923 wait complete. 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.522+0000 7f0c8ab1d640 1 Processor -- start 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.522+0000 7f0c8ab1d640 1 -- start start 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.522+0000 7f0c8ab1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c840cea70 0x7f0c84155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.522+0000 7f0c8ab1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c840d3310 0x7f0c841f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.522+0000 7f0c8ab1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c8415e870 0x7f0c8414e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.522+0000 7f0c8ab1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840cf6d0 con 0x7f0c840d3310 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.522+0000 7f0c8ab1d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84070b60 con 0x7f0c840cea70 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c8ab1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840cf850 con 0x7f0c8415e870 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c840d3310 0x7f0c841f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c888b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c840cea70 0x7f0c84155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c840d3310 0x7f0c841f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34712/0 (socket says 172.21.15.64:34712) 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 -- 172.21.15.64:0/4187228782 learned_addr learned my addr 172.21.15.64:0/4187228782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c890ba640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c8415e870 0x7f0c8414e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 -- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c8415e870 msgr2=0x7f0c8414e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c8415e870 0x7f0c8414e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 -- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c840cea70 msgr2=0x7f0c84155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c840cea70 0x7f0c84155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 -- 172.21.15.64:0/4187228782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c841f4890 con 0x7f0c840d3310 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c890ba640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c8415e870 0x7f0c8414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:29.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c888b9640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c840cea70 0x7f0c84155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:29.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c83fff640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c840d3310 0x7f0c841f4350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c470 tx=0x7f0c70001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c817fa640 1 -- 172.21.15.64:0/4187228782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700052d0 con 0x7f0c840d3310 2024-03-07T07:19:29.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c817fa640 1 -- 172.21.15.64:0/4187228782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70005470 con 0x7f0c840d3310 2024-03-07T07:19:29.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c841f4b60 con 0x7f0c840d3310 2024-03-07T07:19:29.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c817fa640 1 -- 172.21.15.64:0/4187228782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70007260 con 0x7f0c840d3310 2024-03-07T07:19:29.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.523+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c841f5120 con 0x7f0c840d3310 2024-03-07T07:19:29.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.524+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c84062c90 con 0x7f0c840d3310 2024-03-07T07:19:29.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.525+0000 7f0c817fa640 1 -- 172.21.15.64:0/4187228782 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0c70007400 con 0x7f0c840d3310 2024-03-07T07:19:29.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.525+0000 7f0c817fa640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0c6806b4d0 0x7f0c6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.525+0000 7f0c888b9640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0c6806b4d0 0x7f0c6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.525+0000 7f0c817fa640 1 -- 172.21.15.64:0/4187228782 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c70089670 con 0x7f0c840d3310 2024-03-07T07:19:29.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.526+0000 7f0c888b9640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0c6806b4d0 0x7f0c6806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0c78036000 tx=0x7f0c7803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.530+0000 7f0c817fa640 1 -- 172.21.15.64:0/4187228782 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0c70057e90 con 0x7f0c840d3310 2024-03-07T07:19:29.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.713+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c840ca8e0 con 0x7f0c840d3310 2024-03-07T07:19:29.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.713+0000 7f0c817fa640 1 -- 172.21.15.64:0/4187228782 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c7005bf30 con 0x7f0c840d3310 2024-03-07T07:19:29.713 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0c6806b4d0 msgr2=0x7f0c6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0c6806b4d0 0x7f0c6806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0c78036000 tx=0x7f0c7803b040 comp rx=0 tx=0).stop 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c840d3310 msgr2=0x7f0c841f4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c840d3310 0x7f0c841f4350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c470 tx=0x7f0c70001e70 comp rx=0 tx=0).stop 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 shutdown_connections 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0c6806b4d0 0x7f0c6806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c840d3310 0x7f0c841f4350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c8415e870 0x7f0c8414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 --2- 172.21.15.64:0/4187228782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c840cea70 0x7f0c84155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 >> 172.21.15.64:0/4187228782 conn(0x7f0c840d11e0 msgr2=0x7f0c840b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 shutdown_connections 2024-03-07T07:19:29.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.715+0000 7f0c8ab1d640 1 -- 172.21.15.64:0/4187228782 wait complete. 2024-03-07T07:19:29.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.830+0000 7fbb7a761640 1 Processor -- start 2024-03-07T07:19:29.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.830+0000 7fbb7a761640 1 -- start start 2024-03-07T07:19:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb7a761640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb740b7200 0x7fbb74071990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb7a761640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb74071ff0 con 0x7fbb7415e350 2024-03-07T07:19:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb7a761640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740d23b0 con 0x7fbb741625c0 2024-03-07T07:19:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb7a761640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740d2160 con 0x7fbb740b7200 2024-03-07T07:19:29.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb73fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fbb7415e350 0x7fbb74161eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48762/0 (socket says 172.21.15.64:48762) 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb73fff640 1 -- 172.21.15.64:0/2839388252 learned_addr learned my addr 172.21.15.64:0/2839388252 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb78cfe640 1 --2- 172.21.15.64:0/2839388252 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb740b7200 0x7fbb74071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb72ffd640 1 -- 172.21.15.64:0/2839388252 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630752574 0 0) 0x7fbb74071ff0 con 0x7fbb7415e350 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb72ffd640 1 -- 172.21.15.64:0/2839388252 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb5c003520 con 0x7fbb7415e350 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb72ffd640 1 -- 172.21.15.64:0/2839388252 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547826 0 0) 0x7fbb740d23b0 con 0x7fbb741625c0 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb72ffd640 1 -- 172.21.15.64:0/2839388252 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb74071ff0 con 0x7fbb741625c0 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb78cfe640 1 -- 172.21.15.64:0/2839388252 >> v1:172.21.15.120:6789/0 conn(0x7fbb741625c0 legacy=0x7fbb74071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb78cfe640 1 -- 172.21.15.64:0/2839388252 >> v1:172.21.15.64:6789/0 conn(0x7fbb7415e350 legacy=0x7fbb74161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.831+0000 7fbb78cfe640 1 -- 172.21.15.64:0/2839388252 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740d2690 con 0x7fbb740b7200 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb78cfe640 1 --2- 172.21.15.64:0/2839388252 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb740b7200 0x7fbb74071990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009dc0 tx=0x7fbb6c02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=610bd9081d7d801a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb72ffd640 1 -- 172.21.15.64:0/2839388252 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c039070 con 0x7fbb740b7200 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb72ffd640 1 -- 172.21.15.64:0/2839388252 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c02ccd0 con 0x7fbb740b7200 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb72ffd640 1 -- 172.21.15.64:0/2839388252 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c034640 con 0x7fbb740b7200 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb7a761640 1 -- 172.21.15.64:0/2839388252 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb740b7200 msgr2=0x7fbb74071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb7a761640 1 --2- 172.21.15.64:0/2839388252 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb740b7200 0x7fbb74071990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009dc0 tx=0x7fbb6c02bf20 comp rx=0 tx=0).stop 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb7a761640 1 -- 172.21.15.64:0/2839388252 shutdown_connections 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb7a761640 1 --2- 172.21.15.64:0/2839388252 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb740b7200 0x7fbb74071990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.832+0000 7fbb7a761640 1 -- 172.21.15.64:0/2839388252 >> 172.21.15.64:0/2839388252 conn(0x7fbb740d0e70 msgr2=0x7fbb740d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.833+0000 7fbb7a761640 1 -- 172.21.15.64:0/2839388252 shutdown_connections 2024-03-07T07:19:29.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.833+0000 7fbb7a761640 1 -- 172.21.15.64:0/2839388252 wait complete. 2024-03-07T07:19:29.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 Processor -- start 2024-03-07T07:19:29.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 -- start start 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb740b7200 0x7fbb741576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb7415e350 0x7fbb740cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb741625c0 0x7fbb740cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741691e0 con 0x7fbb740b7200 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7415e730 con 0x7fbb741625c0 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.834+0000 7fbb7a761640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74169360 con 0x7fbb7415e350 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb740b7200 0x7fbb741576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb740b7200 0x7fbb741576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34722/0 (socket says 172.21.15.64:34722) 2024-03-07T07:19:29.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 -- 172.21.15.64:0/1406971943 learned_addr learned my addr 172.21.15.64:0/1406971943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb78cfe640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb741625c0 0x7fbb740cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb737fe640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb7415e350 0x7fbb740cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 -- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb7415e350 msgr2=0x7fbb740cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb7415e350 0x7fbb740cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 -- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb741625c0 msgr2=0x7fbb740cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb741625c0 0x7fbb740cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 -- 172.21.15.64:0/1406971943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74143810 con 0x7fbb740b7200 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb73fff640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb740b7200 0x7fbb741576d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c500 tx=0x7fbb6400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb70ff9640 1 -- 172.21.15.64:0/1406971943 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64017070 con 0x7fbb740b7200 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb70ff9640 1 -- 172.21.15.64:0/1406971943 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64005ce0 con 0x7fbb740b7200 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb74143af0 con 0x7fbb740b7200 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb70ff9640 1 -- 172.21.15.64:0/1406971943 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640072f0 con 0x7fbb740b7200 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.835+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb741440b0 con 0x7fbb740b7200 2024-03-07T07:19:29.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.836+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb74062c90 con 0x7fbb740b7200 2024-03-07T07:19:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.837+0000 7fbb70ff9640 1 -- 172.21.15.64:0/1406971943 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbb6401a070 con 0x7fbb740b7200 2024-03-07T07:19:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.837+0000 7fbb70ff9640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb6006b4d0 0x7fbb6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.837+0000 7fbb70ff9640 1 -- 172.21.15.64:0/1406971943 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb64089560 con 0x7fbb740b7200 2024-03-07T07:19:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.837+0000 7fbb737fe640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb6006b4d0 0x7fbb6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:29.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.838+0000 7fbb737fe640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb6006b4d0 0x7fbb6006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb68007fd0 tx=0x7fbb68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:29.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:29.842+0000 7fbb70ff9640 1 -- 172.21.15.64:0/1406971943 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb64057d90 con 0x7fbb740b7200 2024-03-07T07:19:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.020+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb74071060 con 0x7fbb740b7200 2024-03-07T07:19:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.020+0000 7fbb70ff9640 1 -- 172.21.15.64:0/1406971943 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb6405be30 con 0x7fbb740b7200 2024-03-07T07:19:30.020 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb6006b4d0 msgr2=0x7fbb6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb6006b4d0 0x7fbb6006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb68007fd0 tx=0x7fbb68009040 comp rx=0 tx=0).stop 2024-03-07T07:19:30.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb740b7200 msgr2=0x7fbb741576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb740b7200 0x7fbb741576d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c500 tx=0x7fbb6400c9c0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 shutdown_connections 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fbb6006b4d0 0x7fbb6006d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbb741625c0 0x7fbb740cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbb7415e350 0x7fbb740cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 --2- 172.21.15.64:0/1406971943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb740b7200 0x7fbb741576d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 >> 172.21.15.64:0/1406971943 conn(0x7fbb740d0e70 msgr2=0x7fbb740c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 shutdown_connections 2024-03-07T07:19:30.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.022+0000 7fbb7a761640 1 -- 172.21.15.64:0/1406971943 wait complete. 2024-03-07T07:19:30.033 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:30.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.137+0000 7fc771edd640 1 Processor -- start 2024-03-07T07:19:30.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.137+0000 7fc771edd640 1 -- start start 2024-03-07T07:19:30.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc771edd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc76c1625c0 0x7fc76c071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc771edd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c071ff0 con 0x7fc76c15e350 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc771edd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c0d23b0 con 0x7fc76c0b7200 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc771edd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0d2160 con 0x7fc76c1625c0 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc76b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc76c15e350 0x7fc76c161eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:48772/0 (socket says 172.21.15.64:48772) 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc76b7fe640 1 -- 172.21.15.64:0/4187157612 learned_addr learned my addr 172.21.15.64:0/4187157612 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc76affd640 1 --2- 172.21.15.64:0/4187157612 >> v2:172.21.15.120:3300/0 conn(0x7fc76c1625c0 0x7fc76c071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc76a7fc640 1 -- 172.21.15.64:0/4187157612 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298422822 0 0) 0x7fc76c071ff0 con 0x7fc76c15e350 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc76a7fc640 1 -- 172.21.15.64:0/4187157612 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc754003520 con 0x7fc76c15e350 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc76affd640 1 -- 172.21.15.64:0/4187157612 >> v1:172.21.15.120:6789/0 conn(0x7fc76c0b7200 legacy=0x7fc76c0717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.138+0000 7fc76a7fc640 1 -- 172.21.15.64:0/4187157612 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679842269 0 0) 0x7fc76c0d23b0 con 0x7fc76c0b7200 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.139+0000 7fc76affd640 1 -- 172.21.15.64:0/4187157612 >> v1:172.21.15.64:6789/0 conn(0x7fc76c15e350 legacy=0x7fc76c161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.139+0000 7fc76affd640 1 -- 172.21.15.64:0/4187157612 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0d2690 con 0x7fc76c1625c0 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.139+0000 7fc76affd640 1 --2- 172.21.15.64:0/4187157612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c1625c0 0x7fc76c071280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097a0 tx=0x7fc76002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d852274211d98d75 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.139+0000 7fc76a7fc640 1 -- 172.21.15.64:0/4187157612 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760035020 con 0x7fc76c1625c0 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.139+0000 7fc76a7fc640 1 -- 172.21.15.64:0/4187157612 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc760031830 con 0x7fc76c1625c0 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.139+0000 7fc76a7fc640 1 -- 172.21.15.64:0/4187157612 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760031c00 con 0x7fc76c1625c0 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.140+0000 7fc771edd640 1 -- 172.21.15.64:0/4187157612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c1625c0 msgr2=0x7fc76c071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.140+0000 7fc771edd640 1 --2- 172.21.15.64:0/4187157612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c1625c0 0x7fc76c071280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097a0 tx=0x7fc76002b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.140+0000 7fc771edd640 1 -- 172.21.15.64:0/4187157612 shutdown_connections 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.140+0000 7fc771edd640 1 --2- 172.21.15.64:0/4187157612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c1625c0 0x7fc76c071280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.140+0000 7fc771edd640 1 -- 172.21.15.64:0/4187157612 >> 172.21.15.64:0/4187157612 conn(0x7fc76c0d0e70 msgr2=0x7fc76c0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.140+0000 7fc771edd640 1 -- 172.21.15.64:0/4187157612 shutdown_connections 2024-03-07T07:19:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.140+0000 7fc771edd640 1 -- 172.21.15.64:0/4187157612 wait complete. 2024-03-07T07:19:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.141+0000 7fc771edd640 1 Processor -- start 2024-03-07T07:19:30.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.141+0000 7fc771edd640 1 -- start start 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc771edd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c0b7200 0x7fc76c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc771edd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc76c15e350 0x7fc76c15d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc771edd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc76c1625c0 0x7fc76c0740c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc771edd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1691e0 con 0x7fc76c1625c0 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc771edd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c15e730 con 0x7fc76c0b7200 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc771edd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c169360 con 0x7fc76c15e350 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc76c1625c0 0x7fc76c0740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c0b7200 0x7fc76c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc76c1625c0 0x7fc76c0740c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34730/0 (socket says 172.21.15.64:34730) 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 -- 172.21.15.64:0/346536624 learned_addr learned my addr 172.21.15.64:0/346536624 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76affd640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc76c15e350 0x7fc76c15d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 -- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc76c15e350 msgr2=0x7fc76c15d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc76c15e350 0x7fc76c15d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 -- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c0b7200 msgr2=0x7fc76c156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c0b7200 0x7fc76c156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76bfff640 1 -- 172.21.15.64:0/346536624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c15e9e0 con 0x7fc76c1625c0 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.142+0000 7fc76b7fe640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c0b7200 0x7fc76c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.143+0000 7fc76bfff640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc76c1625c0 0x7fc76c0740c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc76400c7b0 tx=0x7fc76400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.143+0000 7fc753fff640 1 -- 172.21.15.64:0/346536624 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764003ba0 con 0x7fc76c1625c0 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.143+0000 7fc753fff640 1 -- 172.21.15.64:0/346536624 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc764003d40 con 0x7fc76c1625c0 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.143+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c15ec80 con 0x7fc76c1625c0 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.143+0000 7fc753fff640 1 -- 172.21.15.64:0/346536624 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764005270 con 0x7fc76c1625c0 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.143+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c15f240 con 0x7fc76c1625c0 2024-03-07T07:19:30.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.144+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c062c90 con 0x7fc76c1625c0 2024-03-07T07:19:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.145+0000 7fc753fff640 1 -- 172.21.15.64:0/346536624 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc764005410 con 0x7fc76c1625c0 2024-03-07T07:19:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.145+0000 7fc753fff640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc75806b4d0 0x7fc75806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.145+0000 7fc753fff640 1 -- 172.21.15.64:0/346536624 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc764089e60 con 0x7fc76c1625c0 2024-03-07T07:19:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.145+0000 7fc76b7fe640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc75806b4d0 0x7fc75806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.145+0000 7fc76b7fe640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc75806b4d0 0x7fc75806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005880 tx=0x7fc75c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.149+0000 7fc753fff640 1 -- 172.21.15.64:0/346536624 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc764058720 con 0x7fc76c1625c0 2024-03-07T07:19:30.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.331+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc76c159f10 con 0x7fc76c1625c0 2024-03-07T07:19:30.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.332+0000 7fc753fff640 1 -- 172.21.15.64:0/346536624 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc76405c7c0 con 0x7fc76c1625c0 2024-03-07T07:19:30.332 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:30.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc75806b4d0 msgr2=0x7fc75806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc75806b4d0 0x7fc75806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005880 tx=0x7fc75c009040 comp rx=0 tx=0).stop 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc76c1625c0 msgr2=0x7fc76c0740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc76c1625c0 0x7fc76c0740c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc76400c7b0 tx=0x7fc76400cbb0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 shutdown_connections 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc75806b4d0 0x7fc75806d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc76c1625c0 0x7fc76c0740c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc76c15e350 0x7fc76c15d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 --2- 172.21.15.64:0/346536624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc76c0b7200 0x7fc76c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 >> 172.21.15.64:0/346536624 conn(0x7fc76c0d0e70 msgr2=0x7fc76c0b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 shutdown_connections 2024-03-07T07:19:30.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.334+0000 7fc771edd640 1 -- 172.21.15.64:0/346536624 wait complete. 2024-03-07T07:19:30.345 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:30.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.446+0000 7f058c189640 1 Processor -- start 2024-03-07T07:19:30.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.446+0000 7f058c189640 1 -- start start 2024-03-07T07:19:30.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f058c189640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f058c189640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0584141c60 con 0x7f05841416d0 2024-03-07T07:19:30.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f058c189640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058415ecd0 con 0x7f0584162030 2024-03-07T07:19:30.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f058c189640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05840b44b0 con 0x7f05840d3ba0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f0589724640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f05841416d0 0x7f05840d3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44844/0 (socket says 172.21.15.64:44844) 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f0589724640 1 -- 172.21.15.64:0/615231724 learned_addr learned my addr 172.21.15.64:0/615231724 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f058a726640 1 --2- 172.21.15.64:0/615231724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f0588f23640 1 -- 172.21.15.64:0/615231724 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922582138 0 0) 0x7f0584141c60 con 0x7f05841416d0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f0588f23640 1 -- 172.21.15.64:0/615231724 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f056c003520 con 0x7f05841416d0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.447+0000 7f0588f23640 1 -- 172.21.15.64:0/615231724 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871063632 0 0) 0x7f058415ecd0 con 0x7f0584162030 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.448+0000 7f0588f23640 1 -- 172.21.15.64:0/615231724 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0584141c60 con 0x7f0584162030 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.448+0000 7f058a726640 1 -- 172.21.15.64:0/615231724 >> v1:172.21.15.120:6789/0 conn(0x7f0584162030 legacy=0x7f0584162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.448+0000 7f058a726640 1 -- 172.21.15.64:0/615231724 >> v1:172.21.15.64:6789/0 conn(0x7f05841416d0 legacy=0x7f05840d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.448+0000 7f058a726640 1 -- 172.21.15.64:0/615231724 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058415efb0 con 0x7f05840d3ba0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.448+0000 7f058a726640 1 --2- 172.21.15.64:0/615231724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058415e680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0574009dc0 tx=0x7f057402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e45a3cfa3098df server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f0588f23640 1 -- 172.21.15.64:0/615231724 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574039070 con 0x7f05840d3ba0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f0588f23640 1 -- 172.21.15.64:0/615231724 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057402ccd0 con 0x7f05840d3ba0 2024-03-07T07:19:30.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f0588f23640 1 -- 172.21.15.64:0/615231724 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574034660 con 0x7f05840d3ba0 2024-03-07T07:19:30.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f058c189640 1 -- 172.21.15.64:0/615231724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 msgr2=0x7f058415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f058c189640 1 --2- 172.21.15.64:0/615231724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058415e680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0574009dc0 tx=0x7f057402bf20 comp rx=0 tx=0).stop 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f058c189640 1 -- 172.21.15.64:0/615231724 shutdown_connections 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f058c189640 1 --2- 172.21.15.64:0/615231724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058415e680 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f058c189640 1 -- 172.21.15.64:0/615231724 >> 172.21.15.64:0/615231724 conn(0x7f0584070400 msgr2=0x7f05840d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f058c189640 1 -- 172.21.15.64:0/615231724 shutdown_connections 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.449+0000 7f058c189640 1 -- 172.21.15.64:0/615231724 wait complete. 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.450+0000 7f058c189640 1 Processor -- start 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.450+0000 7f058c189640 1 -- start start 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058c189640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058413da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058c189640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05841416d0 0x7f0584156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058c189640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0584162030 0x7f0584153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058c189640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058415f570 con 0x7f0584162030 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058c189640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0584162b20 con 0x7f05841416d0 2024-03-07T07:19:30.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058c189640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05840b4940 con 0x7f05840d3ba0 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0584162030 0x7f0584153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0584162030 0x7f0584153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35606/0 (socket says 172.21.15.64:35606) 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f0589724640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05841416d0 0x7f0584156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 -- 172.21.15.64:0/1034169154 learned_addr learned my addr 172.21.15.64:0/1034169154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f0589f25640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058413da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 -- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 msgr2=0x7f058413da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058413da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 -- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05841416d0 msgr2=0x7f0584156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05841416d0 0x7f0584156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f058a726640 1 -- 172.21.15.64:0/1034169154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0584156a60 con 0x7f0584162030 2024-03-07T07:19:30.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.451+0000 7f0589724640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05841416d0 0x7f0584156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.452+0000 7f058a726640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0584162030 0x7f0584153790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0574037040 tx=0x7f0574004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.452+0000 7f05827fc640 1 -- 172.21.15.64:0/1034169154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574039050 con 0x7f0584162030 2024-03-07T07:19:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.452+0000 7f05827fc640 1 -- 172.21.15.64:0/1034169154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574045070 con 0x7f0584162030 2024-03-07T07:19:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.452+0000 7f05827fc640 1 -- 172.21.15.64:0/1034169154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574009450 con 0x7f0584162030 2024-03-07T07:19:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.452+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0584156d40 con 0x7f0584162030 2024-03-07T07:19:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.452+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0584157300 con 0x7f0584162030 2024-03-07T07:19:30.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.453+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0584062c90 con 0x7f0584162030 2024-03-07T07:19:30.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.453+0000 7f05827fc640 1 -- 172.21.15.64:0/1034169154 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f057403e850 con 0x7f0584162030 2024-03-07T07:19:30.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.453+0000 7f05827fc640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f057006b4d0 0x7f057006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.454+0000 7f05827fc640 1 -- 172.21.15.64:0/1034169154 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05740acfa0 con 0x7f0584162030 2024-03-07T07:19:30.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.454+0000 7f0589f25640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f057006b4d0 0x7f057006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.454+0000 7f0589f25640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f057006b4d0 0x7f057006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f057c007fd0 tx=0x7f057c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.458+0000 7f05827fc640 1 -- 172.21.15.64:0/1034169154 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f057407b870 con 0x7f0584162030 2024-03-07T07:19:30.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.673+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05840b8740 con 0x7f0584162030 2024-03-07T07:19:30.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.674+0000 7f05827fc640 1 -- 172.21.15.64:0/1034169154 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7f057407f910 con 0x7f0584162030 2024-03-07T07:19:30.674 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:30.674 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:30.675 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f057006b4d0 msgr2=0x7f057006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f057006b4d0 0x7f057006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f057c007fd0 tx=0x7f057c009040 comp rx=0 tx=0).stop 2024-03-07T07:19:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0584162030 msgr2=0x7f0584153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0584162030 0x7f0584153790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0574037040 tx=0x7f0574004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 shutdown_connections 2024-03-07T07:19:30.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f057006b4d0 0x7f057006d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0584162030 0x7f0584153790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05841416d0 0x7f0584156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 --2- 172.21.15.64:0/1034169154 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05840d3ba0 0x7f058413da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 >> 172.21.15.64:0/1034169154 conn(0x7f0584070400 msgr2=0x7f0584160ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 shutdown_connections 2024-03-07T07:19:30.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.676+0000 7f058c189640 1 -- 172.21.15.64:0/1034169154 wait complete. 2024-03-07T07:19:30.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.788+0000 7f2987006640 1 Processor -- start 2024-03-07T07:19:30.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.788+0000 7f2987006640 1 -- start start 2024-03-07T07:19:30.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f2987006640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f29801417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f2987006640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2980141d10 0x7f29800b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f2987006640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29800b5b00 0x7f2980147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f2987006640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29800b4520 con 0x7f29800b5b00 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f2987006640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29800b4800 con 0x7f2980141d10 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f2987006640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800b4ae0 con 0x7f298015e0d0 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29800b5b00 0x7f2980147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f29800b5b00 0x7f2980147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35612/0 (socket says 172.21.15.64:35612) 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 -- 172.21.15.64:0/2978783133 learned_addr learned my addr 172.21.15.64:0/2978783133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f2984da2640 1 --2- 172.21.15.64:0/2978783133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f29801417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f297ffff640 1 --2- 172.21.15.64:0/2978783133 >> v2:172.21.15.120:3300/0 conn(0x7f2980141d10 0x7f29800b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 -- 172.21.15.64:0/2978783133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 msgr2=0x7f29801417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 --2- 172.21.15.64:0/2978783133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f29801417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 -- 172.21.15.64:0/2978783133 >> v2:172.21.15.120:3300/0 conn(0x7f2980141d10 msgr2=0x7f29800b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 --2- 172.21.15.64:0/2978783133 >> v2:172.21.15.120:3300/0 conn(0x7f2980141d10 0x7f29800b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.789+0000 7f29855a3640 1 -- 172.21.15.64:0/2978783133 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800b4dc0 con 0x7f29800b5b00 2024-03-07T07:19:30.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f29855a3640 1 --2- 172.21.15.64:0/2978783133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29800b5b00 0x7f2980147080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f29800688a0 tx=0x7f297802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21014c87b48a1ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f297f7fe640 1 -- 172.21.15.64:0/2978783133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297802c950 con 0x7f29800b5b00 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f297f7fe640 1 -- 172.21.15.64:0/2978783133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297802caf0 con 0x7f29800b5b00 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f297f7fe640 1 -- 172.21.15.64:0/2978783133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29780343f0 con 0x7f29800b5b00 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 -- 172.21.15.64:0/2978783133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29800b5b00 msgr2=0x7f2980147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 --2- 172.21.15.64:0/2978783133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29800b5b00 0x7f2980147080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f29800688a0 tx=0x7f297802bdb0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 -- 172.21.15.64:0/2978783133 shutdown_connections 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 --2- 172.21.15.64:0/2978783133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29800b5b00 0x7f2980147080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 --2- 172.21.15.64:0/2978783133 >> v2:172.21.15.120:3300/0 conn(0x7f2980141d10 0x7f29800b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 --2- 172.21.15.64:0/2978783133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f29801417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 -- 172.21.15.64:0/2978783133 >> 172.21.15.64:0/2978783133 conn(0x7f2980071300 msgr2=0x7f2980071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 -- 172.21.15.64:0/2978783133 shutdown_connections 2024-03-07T07:19:30.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.790+0000 7f2987006640 1 -- 172.21.15.64:0/2978783133 wait complete. 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.791+0000 7f2987006640 1 Processor -- start 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2987006640 1 -- start start 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2987006640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29800b5b00 0x7f298013e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2987006640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2980141d10 0x7f298012d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2987006640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f2980156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2987006640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29801475c0 con 0x7f2980141d10 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2987006640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298015e4b0 con 0x7f29800b5b00 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2987006640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2980147740 con 0x7f298015e0d0 2024-03-07T07:19:30.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f297ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2980141d10 0x7f298012d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.792+0000 7f2984da2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29800b5b00 0x7f298013e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f297ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2980141d10 0x7f298012d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35614/0 (socket says 172.21.15.64:35614) 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f297ffff640 1 -- 172.21.15.64:0/1868118279 learned_addr learned my addr 172.21.15.64:0/1868118279 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f29855a3640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f2980156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2984da2640 1 -- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 msgr2=0x7f2980156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2984da2640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f2980156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2984da2640 1 -- 172.21.15.64:0/1868118279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2980141d10 msgr2=0x7f298012d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2984da2640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2980141d10 0x7f298012d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2984da2640 1 -- 172.21.15.64:0/1868118279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298013eb70 con 0x7f29800b5b00 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f297ffff640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2980141d10 0x7f298012d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2984da2640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29800b5b00 0x7f298013e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f297400e870 tx=0x7f297400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f297d7fa640 1 -- 172.21.15.64:0/1868118279 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974017070 con 0x7f29800b5b00 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f297d7fa640 1 -- 172.21.15.64:0/1868118279 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400ee60 con 0x7f29800b5b00 2024-03-07T07:19:30.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298012e3f0 con 0x7f29800b5b00 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f297d7fa640 1 -- 172.21.15.64:0/1868118279 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400cb60 con 0x7f29800b5b00 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.793+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2980154380 con 0x7f29800b5b00 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.794+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2980062c90 con 0x7f29800b5b00 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.794+0000 7f297d7fa640 1 -- 172.21.15.64:0/1868118279 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f297401a070 con 0x7f29800b5b00 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.795+0000 7f297d7fa640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f296406b4d0 0x7f296406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.795+0000 7f297d7fa640 1 -- 172.21.15.64:0/1868118279 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2974089f80 con 0x7f29800b5b00 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.795+0000 7f297ffff640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f296406b4d0 0x7f296406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:30.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.795+0000 7f297ffff640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f296406b4d0 0x7f296406d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2980146b20 tx=0x7f296c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:30.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.800+0000 7f297d7fa640 1 -- 172.21.15.64:0/1868118279 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f29740587a0 con 0x7f29800b5b00 2024-03-07T07:19:30.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.985+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29800b7910 con 0x7f29800b5b00 2024-03-07T07:19:30.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.986+0000 7f297d7fa640 1 -- 172.21.15.64:0/1868118279 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f297405c840 con 0x7f29800b5b00 2024-03-07T07:19:30.987 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:30.987 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.988+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f296406b4d0 msgr2=0x7f296406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.988+0000 7f2987006640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f296406b4d0 0x7f296406d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2980146b20 tx=0x7f296c009190 comp rx=0 tx=0).stop 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.988+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29800b5b00 msgr2=0x7f298013e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.988+0000 7f2987006640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29800b5b00 0x7f298013e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f297400e870 tx=0x7f297400ed30 comp rx=0 tx=0).stop 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 shutdown_connections 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f296406b4d0 0x7f296406d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2980141d10 0x7f298012d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f298015e0d0 0x7f2980156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 --2- 172.21.15.64:0/1868118279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29800b5b00 0x7f298013e630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 >> 172.21.15.64:0/1868118279 conn(0x7f2980071300 msgr2=0x7f29800b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 shutdown_connections 2024-03-07T07:19:30.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:30.989+0000 7f2987006640 1 -- 172.21.15.64:0/1868118279 wait complete. 2024-03-07T07:19:31.000 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:31.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.101+0000 7f892b234640 1 Processor -- start 2024-03-07T07:19:31.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.101+0000 7f892b234640 1 -- start start 2024-03-07T07:19:31.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f892b234640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f8924072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f892b234640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89240d1260 0x7f8924072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f892b234640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892415fc30 con 0x7f8924147bb0 2024-03-07T07:19:31.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f892b234640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8924161ce0 con 0x7f89240d1260 2024-03-07T07:19:31.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f892b234640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892415ff10 con 0x7f89240b5090 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89297d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89240d1260 0x7f8924072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89297d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89240d1260 0x7f8924072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35630/0 (socket says 172.21.15.64:35630) 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89297d1640 1 -- 172.21.15.64:0/332518533 learned_addr learned my addr 172.21.15.64:0/332518533 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f8928fd0640 1 --1- 172.21.15.64:0/332518533 >> v1:172.21.15.120:6789/0 conn(0x7f8924147bb0 0x7f89240d0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:54452/0 (socket says 172.21.15.64:54452) 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f8923fff640 1 --2- 172.21.15.64:0/332518533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f8924072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89297d1640 1 -- 172.21.15.64:0/332518533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 msgr2=0x7f8924072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89297d1640 1 --2- 172.21.15.64:0/332518533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f8924072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89297d1640 1 -- 172.21.15.64:0/332518533 >> v1:172.21.15.120:6789/0 conn(0x7f8924147bb0 legacy=0x7f89240d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89297d1640 1 -- 172.21.15.64:0/332518533 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89241601f0 con 0x7f89240d1260 2024-03-07T07:19:31.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.102+0000 7f89237fe640 1 -- 172.21.15.64:0/332518533 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305072898 0 0) 0x7f892415fc30 con 0x7f8924147bb0 2024-03-07T07:19:31.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.103+0000 7f89297d1640 1 --2- 172.21.15.64:0/332518533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240d1260 0x7f8924072e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f891c009e50 tx=0x7f891c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a2ca6b64a5f9fe server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.103+0000 7f89237fe640 1 -- 172.21.15.64:0/332518533 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c02cb00 con 0x7f89240d1260 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.103+0000 7f89237fe640 1 -- 172.21.15.64:0/332518533 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c02cca0 con 0x7f89240d1260 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.103+0000 7f89237fe640 1 -- 172.21.15.64:0/332518533 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c034610 con 0x7f89240d1260 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.103+0000 7f892b234640 1 -- 172.21.15.64:0/332518533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240d1260 msgr2=0x7f8924072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.103+0000 7f892b234640 1 --2- 172.21.15.64:0/332518533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240d1260 0x7f8924072e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f891c009e50 tx=0x7f891c02bf60 comp rx=0 tx=0).stop 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.104+0000 7f892b234640 1 -- 172.21.15.64:0/332518533 shutdown_connections 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.104+0000 7f892b234640 1 --2- 172.21.15.64:0/332518533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89240d1260 0x7f8924072e50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.104+0000 7f892b234640 1 --2- 172.21.15.64:0/332518533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f8924072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.104+0000 7f892b234640 1 -- 172.21.15.64:0/332518533 >> 172.21.15.64:0/332518533 conn(0x7f892406b480 msgr2=0x7f892406b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.104+0000 7f892b234640 1 -- 172.21.15.64:0/332518533 shutdown_connections 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.104+0000 7f892b234640 1 -- 172.21.15.64:0/332518533 wait complete. 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.105+0000 7f892b234640 1 Processor -- start 2024-03-07T07:19:31.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.105+0000 7f892b234640 1 -- start start 2024-03-07T07:19:31.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f892413ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89240d1260 0x7f892413d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924147bb0 0x7f8924141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924147f90 con 0x7f8924147bb0 2024-03-07T07:19:31.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89241607b0 con 0x7f89240d1260 2024-03-07T07:19:31.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8924160cd0 con 0x7f89240b5090 2024-03-07T07:19:31.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924147bb0 0x7f8924141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924147bb0 0x7f8924141ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35646/0 (socket says 172.21.15.64:35646) 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 -- 172.21.15.64:0/3762550576 learned_addr learned my addr 172.21.15.64:0/3762550576 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f8928fd0640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f892413ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f8923fff640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89240d1260 0x7f892413d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 -- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 msgr2=0x7f892413ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f892413ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 -- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89240d1260 msgr2=0x7f892413d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89240d1260 0x7f892413d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 -- 172.21.15.64:0/3762550576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892413c130 con 0x7f8924147bb0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f8923fff640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89240d1260 0x7f892413d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89297d1640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924147bb0 0x7f8924141ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f891c009e50 tx=0x7f891c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89217fa640 1 -- 172.21.15.64:0/3762550576 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c043020 con 0x7f8924147bb0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89217fa640 1 -- 172.21.15.64:0/3762550576 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c02ce50 con 0x7f8924147bb0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892413c3d0 con 0x7f8924147bb0 2024-03-07T07:19:31.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f89217fa640 1 -- 172.21.15.64:0/3762550576 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c009510 con 0x7f8924147bb0 2024-03-07T07:19:31.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.106+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892413c990 con 0x7f8924147bb0 2024-03-07T07:19:31.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.107+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8924062c90 con 0x7f8924147bb0 2024-03-07T07:19:31.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.108+0000 7f89217fa640 1 -- 172.21.15.64:0/3762550576 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f891c046020 con 0x7f8924147bb0 2024-03-07T07:19:31.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.108+0000 7f89217fa640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f890806b500 0x7f890806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.108+0000 7f89217fa640 1 -- 172.21.15.64:0/3762550576 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f891c0ac340 con 0x7f8924147bb0 2024-03-07T07:19:31.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.108+0000 7f8928fd0640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f890806b500 0x7f890806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.109+0000 7f8928fd0640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f890806b500 0x7f890806d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8918007fd0 tx=0x7f8918007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.112+0000 7f89217fa640 1 -- 172.21.15.64:0/3762550576 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f891c04d070 con 0x7f8924147bb0 2024-03-07T07:19:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.337+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f892415ed10 con 0x7f8924147bb0 2024-03-07T07:19:31.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.337+0000 7f89217fa640 1 -- 172.21.15.64:0/3762550576 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7f891c07ac90 con 0x7f8924147bb0 2024-03-07T07:19:31.338 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:31.338 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:31.338 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f890806b500 msgr2=0x7f890806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f890806b500 0x7f890806d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8918007fd0 tx=0x7f8918007de0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924147bb0 msgr2=0x7f8924141ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924147bb0 0x7f8924141ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f891c009e50 tx=0x7f891c004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 shutdown_connections 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f890806b500 0x7f890806d9c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8924147bb0 0x7f8924141ed0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89240d1260 0x7f892413d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 --2- 172.21.15.64:0/3762550576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89240b5090 0x7f892413ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 >> 172.21.15.64:0/3762550576 conn(0x7f892406b480 msgr2=0x7f8924146e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 shutdown_connections 2024-03-07T07:19:31.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.340+0000 7f892b234640 1 -- 172.21.15.64:0/3762550576 wait complete. 2024-03-07T07:19:31.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-07T07:19:15.653340+0000', 'modified': '2024-03-07T07:19:18.012811+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.120:6837/2172286760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6835', 'nonce': 2172286760}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 2172286760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4299, 'qdb_cluster': [4299]} 2024-03-07T07:19:31.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-07T07:19:31.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-07T07:19:31.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.859+0000 7ff534999640 1 Processor -- start 2024-03-07T07:19:31.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.859+0000 7ff534999640 1 -- start start 2024-03-07T07:19:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.859+0000 7ff534999640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5300b5dd0 0x7ff530144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.859+0000 7ff534999640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff530162c80 con 0x7ff530071410 2024-03-07T07:19:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.859+0000 7ff534999640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5300b4bd0 con 0x7ff5300b43f0 2024-03-07T07:19:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.859+0000 7ff534999640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff530141700 con 0x7ff5300b5dd0 2024-03-07T07:19:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52e59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5300b5dd0 0x7ff530144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52ed9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff530071410 0x7ff53015e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44848/0 (socket says 172.21.15.64:44848) 2024-03-07T07:19:31.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52e59c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff5300b5dd0 0x7ff530144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35658/0 (socket says 172.21.15.64:35658) 2024-03-07T07:19:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52ed9d640 1 -- 172.21.15.64:0/1078219479 learned_addr learned my addr 172.21.15.64:0/1078219479 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52e59c640 1 -- 172.21.15.64:0/1078219479 >> v1:172.21.15.120:6789/0 conn(0x7ff5300b43f0 legacy=0x7ff530162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52e59c640 1 -- 172.21.15.64:0/1078219479 >> v1:172.21.15.64:6789/0 conn(0x7ff530071410 legacy=0x7ff53015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52e59c640 1 -- 172.21.15.64:0/1078219479 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300b4eb0 con 0x7ff5300b5dd0 2024-03-07T07:19:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.860+0000 7ff52dd9b640 1 -- 172.21.15.64:0/1078219479 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126253796 0 0) 0x7ff530162c80 con 0x7ff530071410 2024-03-07T07:19:31.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff52e59c640 1 --2- 172.21.15.64:0/1078219479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b5dd0 0x7ff530144540 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5200097a0 tx=0x7ff52002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa711320c17d3d91 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff52dd9b640 1 -- 172.21.15.64:0/1078219479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520035020 con 0x7ff5300b5dd0 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff52dd9b640 1 -- 172.21.15.64:0/1078219479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520031830 con 0x7ff5300b5dd0 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff52dd9b640 1 -- 172.21.15.64:0/1078219479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520031c00 con 0x7ff5300b5dd0 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff534999640 1 -- 172.21.15.64:0/1078219479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b5dd0 msgr2=0x7ff530144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff534999640 1 --2- 172.21.15.64:0/1078219479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b5dd0 0x7ff530144540 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5200097a0 tx=0x7ff52002b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff534999640 1 -- 172.21.15.64:0/1078219479 shutdown_connections 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff534999640 1 --2- 172.21.15.64:0/1078219479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b5dd0 0x7ff530144540 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff534999640 1 -- 172.21.15.64:0/1078219479 >> 172.21.15.64:0/1078219479 conn(0x7ff5300d0e70 msgr2=0x7ff5300d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff534999640 1 -- 172.21.15.64:0/1078219479 shutdown_connections 2024-03-07T07:19:31.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.861+0000 7ff534999640 1 -- 172.21.15.64:0/1078219479 wait complete. 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 Processor -- start 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 -- start start 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530071410 0x7ff530141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b43f0 0x7ff53015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5300b5dd0 0x7ff530161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff530162050 con 0x7ff5300b43f0 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff530161e00 con 0x7ff530071410 2024-03-07T07:19:31.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff534999640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5300b61b0 con 0x7ff5300b5dd0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.863+0000 7ff527fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b43f0 0x7ff53015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b43f0 0x7ff53015b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35670/0 (socket says 172.21.15.64:35670) 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 -- 172.21.15.64:0/8346579 learned_addr learned my addr 172.21.15.64:0/8346579 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff52e59c640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530071410 0x7ff530141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5300b5dd0 0x7ff530161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 -- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5300b5dd0 msgr2=0x7ff530161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5300b5dd0 0x7ff530161070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 -- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530071410 msgr2=0x7ff530141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530071410 0x7ff530141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 -- 172.21.15.64:0/8346579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5301e96a0 con 0x7ff5300b43f0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff527fff640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b43f0 0x7ff53015b910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c480 tx=0x7ff51800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff526ffd640 1 -- 172.21.15.64:0/8346579 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5180052d0 con 0x7ff5300b43f0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff526ffd640 1 -- 172.21.15.64:0/8346579 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff518005470 con 0x7ff5300b43f0 2024-03-07T07:19:31.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5301e9920 con 0x7ff5300b43f0 2024-03-07T07:19:31.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff526ffd640 1 -- 172.21.15.64:0/8346579 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518012870 con 0x7ff5300b43f0 2024-03-07T07:19:31.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.864+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5301e9ee0 con 0x7ff5300b43f0 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.866+0000 7ff526ffd640 1 -- 172.21.15.64:0/8346579 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff518012a10 con 0x7ff5300b43f0 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.866+0000 7ff526ffd640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff51006b4d0 0x7ff51006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.866+0000 7ff526ffd640 1 -- 172.21.15.64:0/8346579 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff518089cb0 con 0x7ff5300b43f0 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.866+0000 7ff52e59c640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff51006b4d0 0x7ff51006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.867+0000 7ff52e59c640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff51006b4d0 0x7ff51006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff520036000 tx=0x7ff520008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.867+0000 7ff537778740 1 Processor -- start 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.867+0000 7ff537778740 1 -- start start 2024-03-07T07:19:31.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff537778740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb7b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb7b1620 0x5588eb7895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb789b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff537778740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eb6ee790 con 0x5588eb78a5b0 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eb697b20 con 0x5588eb78a990 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588eb6ea830 con 0x5588eb7b1620 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb7b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb7b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35674/0 (socket says 172.21.15.64:35674) 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 -- 172.21.15.64:0/3866912473 learned_addr learned my addr 172.21.15.64:0/3866912473 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff527fff640 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb7b1620 0x5588eb7895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb789b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 -- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb7b1620 msgr2=0x5588eb7895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb7b1620 0x5588eb7895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 -- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 msgr2=0x5588eb789b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb789b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff52e59c640 1 -- 172.21.15.64:0/3866912473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588eb6ab970 con 0x5588eb78a5b0 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.868+0000 7ff527fff640 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb7b1620 0x5588eb7895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff52e59c640 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb7b10e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c75c8f06015915 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff50ffff640 1 -- 172.21.15.64:0/3866912473 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520035040 con 0x5588eb78a5b0 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff50ffff640 1 -- 172.21.15.64:0/3866912473 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520003750 con 0x5588eb78a5b0 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff50ffff640 1 -- 172.21.15.64:0/3866912473 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520003b20 con 0x5588eb78a5b0 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff537778740 1 -- 172.21.15.64:0/3866912473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 msgr2=0x5588eb7b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff537778740 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb7b10e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520030000 comp rx=0 tx=0).stop 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff537778740 1 -- 172.21.15.64:0/3866912473 shutdown_connections 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff537778740 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb789b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff537778740 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb7b1620 0x5588eb7895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff537778740 1 --2- 172.21.15.64:0/3866912473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb7b10e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.869+0000 7ff537778740 1 -- 172.21.15.64:0/3866912473 >> 172.21.15.64:0/3866912473 conn(0x5588eb7513d0 msgr2=0x5588eb7cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.870+0000 7ff537778740 1 -- 172.21.15.64:0/3866912473 shutdown_connections 2024-03-07T07:19:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.870+0000 7ff537778740 1 -- 172.21.15.64:0/3866912473 wait complete. 2024-03-07T07:19:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.870+0000 7ff537778740 1 -- start start 2024-03-07T07:19:31.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.870+0000 7ff537778740 1 Processor -- start 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb7a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff537778740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb7cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff537778740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eb7b0c50 con 0x5588eb7cbb40 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eb6aaf60 con 0x5588eb78a990 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588eb7a99d0 con 0x5588eb78a5b0 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff537778740 10 client.0 _ll_register_callbacks cb 0x5588eb6e63d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb7cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb7cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35690/0 (socket says 172.21.15.64:35690) 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 -- 172.21.15.64:0/1561387893 learned_addr learned my addr 172.21.15.64:0/1561387893 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52e59c640 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb7a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff527fff640 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 -- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 msgr2=0x5588eb7a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb7a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 -- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 msgr2=0x5588eb7b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.871+0000 7ff52ed9d640 1 -- 172.21.15.64:0/1561387893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588eb4032d0 con 0x5588eb7cbb40 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb7cbfa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5280072d0 tx=0x7ff5280076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59132db68ff0baa server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff50d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff50d7fa640 1 -- 172.21.15.64:0/1561387893 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528007a10 con 0x5588eb7cbb40 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff50d7fa640 1 -- 172.21.15.64:0/1561387893 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff528012820 con 0x5588eb7cbb40 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff520003750 con 0x5588eb7cbb40 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff50d7fa640 1 -- 172.21.15.64:0/1561387893 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff528022050 con 0x5588eb7cbb40 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 10 client.4650 fetch_fsmap learned FSMap version 7 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5588eb4cd6b0 con 0x5588eb7cbb40 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff50d7fa640 1 -- 172.21.15.64:0/1561387893 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520003750 con 0x5588eb7cbb40 2024-03-07T07:19:31.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff50d7fa640 1 -- 172.21.15.64:0/1561387893 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 7) v1 ==== 3314+0+0 (secure 0 0 0) 0x7ff528025020 con 0x5588eb7cbb40 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 10 client.4650 fetch_fsmap finished waiting for FSMap version 7 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 10 client.4650 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 20 client.4650 populate_metadata read hostname 'smithi064' 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7b59d0 0x5588eb7b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 4 client.4650 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 --> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588eb6fbcd0 con 0x5588eb7b59d0 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.872+0000 7ff52e59c640 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7b59d0 0x5588eb7b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.873+0000 7ff52e59c640 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7b59d0 0x5588eb7b5db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e5be4d02ce3ab3 server_cookie=7e76e799e6e48a7b in_seq=0 out_seq=0 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.873+0000 7ff50d7fa640 10 client.4650 ms_handle_connect on v2:172.21.15.120:6835/2172286760 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.873+0000 7ff50d7fa640 1 -- 172.21.15.64:0/1561387893 <== mds.0 v2:172.21.15.120:6835/2172286760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5588eb6fbcd0 con 0x5588eb7b59d0 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.873+0000 7ff50d7fa640 10 client.4650 handle_command_reply: tid=0 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.873+0000 7ff537778740 1 client.4650 shutdown 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 msgr2=0x5588eb7cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb7cbfa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5280072d0 tx=0x7ff5280076d0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 shutdown_connections 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb7cbfa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7b59d0 0x5588eb7b5db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff50d7fa640 0 client.4650 ms_handle_reset on v2:172.21.15.120:6835/2172286760 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 --2- 172.21.15.64:0/1561387893 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb7a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 >> 172.21.15.64:0/1561387893 conn(0x5588eb7513d0 msgr2=0x5588eb7b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 shutdown_connections 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 1 -- 172.21.15.64:0/1561387893 wait complete. 2024-03-07T07:19:31.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.874+0000 7ff537778740 20 client.4650 trim_cache size 0 max 16384 2024-03-07T07:19:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 Processor -- start 2024-03-07T07:19:31.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 -- start start 2024-03-07T07:19:31.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb839390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb783890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5301ff3a0 con 0x5588eb7cbb40 2024-03-07T07:19:31.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53015f3e0 con 0x5588eb78a990 2024-03-07T07:19:31.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.891+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5301fedd0 con 0x5588eb78a5b0 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb783890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb783890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35696/0 (socket says 172.21.15.64:35696) 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 -- 172.21.15.64:0/871096355 learned_addr learned my addr 172.21.15.64:0/871096355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff527fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52e59c640 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb839390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 -- 172.21.15.64:0/871096355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 msgr2=0x5588eb839390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb839390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 -- 172.21.15.64:0/871096355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 msgr2=0x5588eb7b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 -- 172.21.15.64:0/871096355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588eb7a8c80 con 0x5588eb7cbb40 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb783890 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff528002090 tx=0x7ff528006030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31bb5e6f41b9547 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff50f7fe640 1 -- 172.21.15.64:0/871096355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52802a070 con 0x5588eb7cbb40 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff50f7fe640 1 -- 172.21.15.64:0/871096355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52800b040 con 0x5588eb7cbb40 2024-03-07T07:19:31.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.892+0000 7ff50f7fe640 1 -- 172.21.15.64:0/871096355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528012b60 con 0x5588eb7cbb40 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 -- 172.21.15.64:0/871096355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 msgr2=0x5588eb783890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb783890 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff528002090 tx=0x7ff528006030 comp rx=0 tx=0).stop 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 -- 172.21.15.64:0/871096355 shutdown_connections 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb7cbb40 0x5588eb783890 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb78a990 0x5588eb7b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 --2- 172.21.15.64:0/871096355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a5b0 0x5588eb839390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 -- 172.21.15.64:0/871096355 >> 172.21.15.64:0/871096355 conn(0x5588eb7513d0 msgr2=0x5588eb7b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 -- 172.21.15.64:0/871096355 shutdown_connections 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 -- 172.21.15.64:0/871096355 wait complete. 2024-03-07T07:19:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 -- start start 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.893+0000 7ff537778740 1 Processor -- start 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff537778740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb837e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a990 0x5588eb7f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff537778740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb7cbb40 0x5588eb7d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff537778740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eb7cd9a0 con 0x5588eb78a5b0 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588eb7a8960 con 0x5588eb7cbb40 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff537778740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588eb7cd180 con 0x5588eb78a990 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff537778740 10 client.0 _ll_register_callbacks cb 0x7ff528004da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb837e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff52e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb837e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35700/0 (socket says 172.21.15.64:35700) 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff52e59c640 1 -- 172.21.15.64:0/868054847 learned_addr learned my addr 172.21.15.64:0/868054847 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff52ed9d640 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb7cbb40 0x5588eb7d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.894+0000 7ff527fff640 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a990 0x5588eb7f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff52e59c640 1 -- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a990 msgr2=0x5588eb7f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff52e59c640 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a990 0x5588eb7f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff52e59c640 1 -- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb7cbb40 msgr2=0x5588eb7d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff52e59c640 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb7cbb40 0x5588eb7d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff52e59c640 1 -- 172.21.15.64:0/868054847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588eb7c9370 con 0x5588eb78a5b0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff52e59c640 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb837e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b674e0080c6fa2a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff50d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff50d7fa640 1 -- 172.21.15.64:0/868054847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52005f070 con 0x5588eb78a5b0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff50d7fa640 1 -- 172.21.15.64:0/868054847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5200042c0 con 0x5588eb78a5b0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52800b040 con 0x5588eb78a5b0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff50d7fa640 1 -- 172.21.15.64:0/868054847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff52006a050 con 0x5588eb78a5b0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 10 client.4656 fetch_fsmap learned FSMap version 7 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5588eb7d2e50 con 0x5588eb78a5b0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff50d7fa640 1 -- 172.21.15.64:0/868054847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52800b040 con 0x5588eb78a5b0 2024-03-07T07:19:31.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff50d7fa640 1 -- 172.21.15.64:0/868054847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 7) v1 ==== 3314+0+0 (secure 0 0 0) 0x7ff52006d020 con 0x5588eb78a5b0 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 10 client.4656 fetch_fsmap finished waiting for FSMap version 7 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 20 client.4656 populate_metadata read hostname 'smithi064' 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7cb170 0x5588eb7b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 4 client.4656 mds_command: new command op to 4299 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.895+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 --> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5588eb6f6020 con 0x5588eb7cb170 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff527fff640 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7cb170 0x5588eb7b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff527fff640 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7cb170 0x5588eb7b3e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52216426d41d16c server_cookie=224746d6e1dae3f6 in_seq=0 out_seq=0 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff50d7fa640 10 client.4656 ms_handle_connect on v2:172.21.15.120:6835/2172286760 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff50d7fa640 1 -- 172.21.15.64:0/868054847 <== mds.0 v2:172.21.15.120:6835/2172286760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x5588eb6f6020 con 0x5588eb7cb170 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff50d7fa640 10 client.4656 handle_command_reply: tid=0 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 client.4656 shutdown 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 msgr2=0x5588eb837e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb837e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004730 comp rx=0 tx=0).stop 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 shutdown_connections 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:6835/2172286760,v1:172.21.15.120:6837/2172286760] conn(0x5588eb7cb170 0x5588eb7b3e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588eb7cbb40 0x5588eb7d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5588eb78a990 0x5588eb7f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff50d7fa640 0 client.4656 ms_handle_reset on v2:172.21.15.120:6835/2172286760 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 --2- 172.21.15.64:0/868054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5588eb78a5b0 0x5588eb837e90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 >> 172.21.15.64:0/868054847 conn(0x5588eb7513d0 msgr2=0x5588eb7a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 shutdown_connections 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 1 -- 172.21.15.64:0/868054847 wait complete. 2024-03-07T07:19:31.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.896+0000 7ff537778740 20 client.4656 trim_cache size 0 max 16384 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3", 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 4299, 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 15.607499119, 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 7, 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 20, 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 20, 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 22.153274952 2024-03-07T07:19:31.905 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:19:31.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff51006b4d0 msgr2=0x7ff51006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff51006b4d0 0x7ff51006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff520036000 tx=0x7ff520008030 comp rx=0 tx=0).stop 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b43f0 msgr2=0x7ff53015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b43f0 0x7ff53015b910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c480 tx=0x7ff51800c940 comp rx=0 tx=0).stop 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 shutdown_connections 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff51006b4d0 0x7ff51006d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5300b43f0 0x7ff53015b910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5300b5dd0 0x7ff530161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 --2- 172.21.15.64:0/8346579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff530071410 0x7ff530141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 >> 172.21.15.64:0/8346579 conn(0x7ff5300d0e70 msgr2=0x7ff530143cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 shutdown_connections 2024-03-07T07:19:31.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:31.897+0000 7ff534999640 1 -- 172.21.15.64:0/8346579 wait complete. 2024-03-07T07:19:31.909 INFO:teuthology.run_tasks:Running task kclient... 2024-03-07T07:19:31.935 INFO:tasks.kclient:Mounting kernel clients... 2024-03-07T07:19:31.935 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-07T07:19:31.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:31.935 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-03-07T07:19:31.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:31.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-03-07T07:19:32.018 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-03-07T07:19:32.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:19:32.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.020 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-03-07T07:19:32.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.074 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-03-07T07:19:32.142 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-03-07T07:19:32.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:19:32.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.144 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-03-07T07:19:32.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.158 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-03-07T07:19:32.227 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-03-07T07:19:32.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:19:32.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.229 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-03-07T07:19:32.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.285 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-03-07T07:19:32.352 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-03-07T07:19:32.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:19:32.354 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-07T07:19:32.354 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T07:19:32.354 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T07:19:32.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T07:19:32.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.457+0000 7f5f69034640 1 Processor -- start 2024-03-07T07:19:32.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.457+0000 7f5f69034640 1 -- start start 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f69034640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f64147790 0x7f5f640b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f69034640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f640b4520 con 0x7f5f640b55c0 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f69034640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f640b4800 con 0x7f5f6415e0d0 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f69034640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f640b5f60 con 0x7f5f64147790 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f6259c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5f640b55c0 0x7f5f64147080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44854/0 (socket says 172.21.15.64:44854) 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f6259c640 1 -- 172.21.15.64:0/3133700920 learned_addr learned my addr 172.21.15.64:0/3133700920 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f6359e640 1 --2- 172.21.15.64:0/3133700920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f64147790 0x7f5f640b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f61d9b640 1 -- 172.21.15.64:0/3133700920 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702882181 0 0) 0x7f5f640b4520 con 0x7f5f640b55c0 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f61d9b640 1 -- 172.21.15.64:0/3133700920 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f50003520 con 0x7f5f640b55c0 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f61d9b640 1 -- 172.21.15.64:0/3133700920 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787349115 0 0) 0x7f5f640b4800 con 0x7f5f6415e0d0 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f61d9b640 1 -- 172.21.15.64:0/3133700920 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f640b4520 con 0x7f5f6415e0d0 2024-03-07T07:19:32.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f6359e640 1 -- 172.21.15.64:0/3133700920 >> v1:172.21.15.120:6789/0 conn(0x7f5f6415e0d0 legacy=0x7f5f641417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f6359e640 1 -- 172.21.15.64:0/3133700920 >> v1:172.21.15.64:6789/0 conn(0x7f5f640b55c0 legacy=0x7f5f64147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.458+0000 7f5f6359e640 1 -- 172.21.15.64:0/3133700920 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f640b4ae0 con 0x7f5f64147790 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.459+0000 7f5f6359e640 1 --2- 172.21.15.64:0/3133700920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f64147790 0x7f5f640b5a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009dc0 tx=0x7f5f5402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a468ba2a9d8c459e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.459+0000 7f5f61d9b640 1 -- 172.21.15.64:0/3133700920 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54039070 con 0x7f5f64147790 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.459+0000 7f5f61d9b640 1 -- 172.21.15.64:0/3133700920 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5402cbb0 con 0x7f5f64147790 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.459+0000 7f5f61d9b640 1 -- 172.21.15.64:0/3133700920 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54034440 con 0x7f5f64147790 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.460+0000 7f5f69034640 1 -- 172.21.15.64:0/3133700920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f64147790 msgr2=0x7f5f640b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.460+0000 7f5f69034640 1 --2- 172.21.15.64:0/3133700920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f64147790 0x7f5f640b5a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009dc0 tx=0x7f5f5402be00 comp rx=0 tx=0).stop 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.460+0000 7f5f69034640 1 -- 172.21.15.64:0/3133700920 shutdown_connections 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.460+0000 7f5f69034640 1 --2- 172.21.15.64:0/3133700920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f64147790 0x7f5f640b5a20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.460+0000 7f5f69034640 1 -- 172.21.15.64:0/3133700920 >> 172.21.15.64:0/3133700920 conn(0x7f5f64071300 msgr2=0x7f5f64071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.460+0000 7f5f69034640 1 -- 172.21.15.64:0/3133700920 shutdown_connections 2024-03-07T07:19:32.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.460+0000 7f5f69034640 1 -- 172.21.15.64:0/3133700920 wait complete. 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.461+0000 7f5f69034640 1 Processor -- start 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.461+0000 7f5f69034640 1 -- start start 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f69034640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f640b55c0 0x7f5f64155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f69034640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f64147790 0x7f5f64140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f69034640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f6415e0d0 0x7f5f64159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f69034640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f64141ee0 con 0x7f5f64147790 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f69034640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f640b50a0 con 0x7f5f640b55c0 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f69034640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6415e4b0 con 0x7f5f6415e0d0 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f64147790 0x7f5f64140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:32.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f64147790 0x7f5f64140c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35708/0 (socket says 172.21.15.64:35708) 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 -- 172.21.15.64:0/113078070 learned_addr learned my addr 172.21.15.64:0/113078070 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f62d9d640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f640b55c0 0x7f5f64155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6359e640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f6415e0d0 0x7f5f64159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 -- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f6415e0d0 msgr2=0x7f5f64159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f6415e0d0 0x7f5f64159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 -- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f640b55c0 msgr2=0x7f5f64155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f640b55c0 0x7f5f64155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 -- 172.21.15.64:0/113078070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f641566c0 con 0x7f5f64147790 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f6259c640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f64147790 0x7f5f64140c60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00c3e0 tx=0x7f5f4c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f5b7fe640 1 -- 172.21.15.64:0/113078070 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c017070 con 0x7f5f64147790 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.462+0000 7f5f5b7fe640 1 -- 172.21.15.64:0/113078070 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c005d00 con 0x7f5f64147790 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.463+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f641569a0 con 0x7f5f64147790 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.463+0000 7f5f5b7fe640 1 -- 172.21.15.64:0/113078070 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c007310 con 0x7f5f64147790 2024-03-07T07:19:32.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.463+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f64156f60 con 0x7f5f64147790 2024-03-07T07:19:32.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.463+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f64062c90 con 0x7f5f64147790 2024-03-07T07:19:32.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.464+0000 7f5f5b7fe640 1 -- 172.21.15.64:0/113078070 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5f4c01a070 con 0x7f5f64147790 2024-03-07T07:19:32.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.464+0000 7f5f5b7fe640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5f4406b4d0 0x7f5f4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:32.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.464+0000 7f5f5b7fe640 1 -- 172.21.15.64:0/113078070 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f4c089520 con 0x7f5f64147790 2024-03-07T07:19:32.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.464+0000 7f5f62d9d640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5f4406b4d0 0x7f5f4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:32.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.465+0000 7f5f62d9d640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5f4406b4d0 0x7f5f4406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0011e0 tx=0x7f5f5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:32.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.469+0000 7f5f5b7fe640 1 -- 172.21.15.64:0/113078070 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5f4c057ea0 con 0x7f5f64147790 2024-03-07T07:19:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.693+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5f641472b0 con 0x7f5f64147790 2024-03-07T07:19:32.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.693+0000 7f5f5b7fe640 1 -- 172.21.15.64:0/113078070 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5f4c05bf40 con 0x7f5f64147790 2024-03-07T07:19:32.693 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T07:19:32.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5f4406b4d0 msgr2=0x7f5f4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:32.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5f4406b4d0 0x7f5f4406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0011e0 tx=0x7f5f5c009040 comp rx=0 tx=0).stop 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f64147790 msgr2=0x7f5f64140c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f64147790 0x7f5f64140c60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00c3e0 tx=0x7f5f4c00c8a0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 shutdown_connections 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5f4406b4d0 0x7f5f4406d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f64147790 0x7f5f64140c60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5f6415e0d0 0x7f5f64159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 --2- 172.21.15.64:0/113078070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f640b55c0 0x7f5f64155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 >> 172.21.15.64:0/113078070 conn(0x7f5f64071300 msgr2=0x7f5f64144ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 shutdown_connections 2024-03-07T07:19:32.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:32.695+0000 7f5f69034640 1 -- 172.21.15.64:0/113078070 wait complete. 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T07:19:32.706 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T07:19:32.707 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T07:19:32.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.707 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:32.722 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42704sec preferred_lft 42704sec 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-03-07T07:19:32.723 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:32.723 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-07T07:19:32.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.724 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:32.724 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:32.724 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-03-07T07:19:32.724 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-03-07T07:19:32.724 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-03-07T07:19:32.724 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-07T07:19:32.724 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:32.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.887 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T07:19:32.913 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-03-07T07:19:32.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.914 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-03-07T07:19:32.979 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-03-07T07:19:32.979 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T07:19:32.979 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T07:19:32.979 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T07:19:32.979 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T07:19:32.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:32.980 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:32.980 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:32.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T07:19:32.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T07:19:32.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T07:19:32.980 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:33.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.229 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-03-07T07:19:33.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.244 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-03-07T07:19:33.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.300 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:33.300 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:33.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T07:19:33.300 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-07T07:19:33.300 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:33.384 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-07T07:19:33.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.385 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:33.385 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:33.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-07T07:19:33.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-07T07:19:33.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-07T07:19:33.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-07T07:19:33.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-07T07:19:33.386 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:33.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.542 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:33.542 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:33.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-03-07T07:19:33.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-03-07T07:19:33.542 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:33.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:33.584 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-07T07:19:33.636 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-07T07:19:33.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:33.636 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-07T07:19:33.692 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-07T07:19:33.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.692 DEBUG:teuthology.orchestra.run.smithi064:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:33.777 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:33.777 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:33.890 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:33.890 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:33.890 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: resolved to: "172.21.15.64:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-03-07T07:19:33.890 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: trying mount with new device syntax: 0@6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.cephfs=/ 2024-03-07T07:19:33.890 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.64:3300/172.21.15.120:3300/172.21.15.120:3301" will pass to kernel 2024-03-07T07:19:33.892 INFO:teuthology.orchestra.run.smithi064.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-07T07:19:33.892 INFO:teuthology.orchestra.run.smithi064.stdout: You just mounted a file system that supports labels which does not 2024-03-07T07:19:33.892 INFO:teuthology.orchestra.run.smithi064.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T07:19:33.892 INFO:teuthology.orchestra.run.smithi064.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T07:19:33.892 INFO:teuthology.orchestra.run.smithi064.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T07:19:33.894 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T07:19:33.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.894 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-07T07:19:33.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:33.922 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:33.982 INFO:teuthology.orchestra.run.smithi064.stdout:client4663 2024-03-07T07:19:33.982 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:33.982 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:34.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.037 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:34.096 INFO:teuthology.orchestra.run.smithi064.stdout:client4663 2024-03-07T07:19:34.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.096 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:34.156 INFO:teuthology.orchestra.run.smithi064.stdout:client4663 2024-03-07T07:19:34.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.157 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.client4663/status') 2024-03-07T07:19:34.226 INFO:teuthology.orchestra.run.smithi064.stdout:instance: client.4663 (3)172.21.15.64:0/1413427260 2024-03-07T07:19:34.226 INFO:teuthology.orchestra.run.smithi064.stdout:blocklisted: false 2024-03-07T07:19:34.226 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records in 2024-03-07T07:19:34.227 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records out 2024-03-07T07:19:34.227 INFO:teuthology.orchestra.run.smithi064.stderr:70 bytes copied, 2.0306e-05 s, 3.4 MB/s 2024-03-07T07:19:34.228 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-07T07:19:34.228 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T07:19:34.228 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-07T07:19:34.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T07:19:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.371+0000 7ff1ce4d4640 1 Processor -- start 2024-03-07T07:19:34.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.371+0000 7ff1ce4d4640 1 -- start start 2024-03-07T07:19:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1ce4d4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c8070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1ce4d4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c8147ea0 con 0x7ff1c80b6120 2024-03-07T07:19:34.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1ce4d4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c80b4960 con 0x7ff1c80ae470 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1ce4d4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c8071390 con 0x7ff1c80b43f0 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1c7fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff1c80b6120 0x7ff1c80b6500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44862/0 (socket says 172.21.15.64:44862) 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1c7fff640 1 -- 172.21.15.64:0/3212965960 learned_addr learned my addr 172.21.15.64:0/3212965960 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1c77fe640 1 --2- 172.21.15.64:0/3212965960 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c8070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1c6ffd640 1 -- 172.21.15.64:0/3212965960 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658501314 0 0) 0x7ff1bc001110 con 0x7ff1c80b6120 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1c6ffd640 1 -- 172.21.15.64:0/3212965960 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b0003520 con 0x7ff1c80b6120 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1c6ffd640 1 -- 172.21.15.64:0/3212965960 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118492170 0 0) 0x7ff1c80b4960 con 0x7ff1c80ae470 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.372+0000 7ff1c6ffd640 1 -- 172.21.15.64:0/3212965960 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1bc001110 con 0x7ff1c80ae470 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.373+0000 7ff1c77fe640 1 -- 172.21.15.64:0/3212965960 >> v1:172.21.15.120:6789/0 conn(0x7ff1c80ae470 legacy=0x7ff1c80ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.373+0000 7ff1c77fe640 1 -- 172.21.15.64:0/3212965960 >> v1:172.21.15.64:6789/0 conn(0x7ff1c80b6120 legacy=0x7ff1c80b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.373+0000 7ff1c77fe640 1 -- 172.21.15.64:0/3212965960 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80b4c40 con 0x7ff1c80b43f0 2024-03-07T07:19:34.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.373+0000 7ff1c77fe640 1 --2- 172.21.15.64:0/3212965960 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c8070e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80097d0 tx=0x7ff1b802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9318cddb36853661 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.373+0000 7ff1c6ffd640 1 -- 172.21.15.64:0/3212965960 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8035020 con 0x7ff1c80b43f0 2024-03-07T07:19:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1c6ffd640 1 -- 172.21.15.64:0/3212965960 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8031890 con 0x7ff1c80b43f0 2024-03-07T07:19:34.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1c6ffd640 1 -- 172.21.15.64:0/3212965960 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8031c40 con 0x7ff1c80b43f0 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3212965960 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 msgr2=0x7ff1c8070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3212965960 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c8070e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80097d0 tx=0x7ff1b802b930 comp rx=0 tx=0).stop 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3212965960 shutdown_connections 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3212965960 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c8070e50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3212965960 >> 172.21.15.64:0/3212965960 conn(0x7ff1c8071550 msgr2=0x7ff1c8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3212965960 shutdown_connections 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.374+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3212965960 wait complete. 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.375+0000 7ff1ce4d4640 1 Processor -- start 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.375+0000 7ff1ce4d4640 1 -- start start 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1ce4d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c80ae470 0x7ff1c815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1ce4d4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c80b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:34.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1ce4d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c80b6120 0x7ff1c8142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1ce4d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80aefe0 con 0x7ff1c80b6120 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1ce4d4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80b5200 con 0x7ff1c80ae470 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1ce4d4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80af160 con 0x7ff1c80b43f0 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1cca71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c80b6120 0x7ff1c8142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1cca71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c80b6120 0x7ff1c8142cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35714/0 (socket says 172.21.15.64:35714) 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1cca71640 1 -- 172.21.15.64:0/3887453260 learned_addr learned my addr 172.21.15.64:0/3887453260 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1c7fff640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c80ae470 0x7ff1c815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.376+0000 7ff1c77fe640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c80b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1cca71640 1 -- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 msgr2=0x7ff1c80b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1cca71640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c80b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1cca71640 1 -- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c80ae470 msgr2=0x7ff1c815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1cca71640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c80ae470 0x7ff1c815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1cca71640 1 -- 172.21.15.64:0/3887453260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80b0c10 con 0x7ff1c80b6120 2024-03-07T07:19:34.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1cca71640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c80b6120 0x7ff1c8142cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c7c0 tx=0x7ff1c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1c4ff9640 1 -- 172.21.15.64:0/3887453260 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0003b70 con 0x7ff1c80b6120 2024-03-07T07:19:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1c4ff9640 1 -- 172.21.15.64:0/3887453260 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c0003d10 con 0x7ff1c80b6120 2024-03-07T07:19:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c80b0ef0 con 0x7ff1c80b6120 2024-03-07T07:19:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1c4ff9640 1 -- 172.21.15.64:0/3887453260 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000aa50 con 0x7ff1c80b6120 2024-03-07T07:19:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.377+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c80b22a0 con 0x7ff1c80b6120 2024-03-07T07:19:34.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.378+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c8062c90 con 0x7ff1c80b6120 2024-03-07T07:19:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.378+0000 7ff1c4ff9640 1 -- 172.21.15.64:0/3887453260 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff1c000abf0 con 0x7ff1c80b6120 2024-03-07T07:19:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.379+0000 7ff1c4ff9640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff1b406b4d0 0x7ff1b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.379+0000 7ff1c4ff9640 1 -- 172.21.15.64:0/3887453260 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1c008a060 con 0x7ff1c80b6120 2024-03-07T07:19:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.379+0000 7ff1c7fff640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff1b406b4d0 0x7ff1b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:34.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.379+0000 7ff1c7fff640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff1b406b4d0 0x7ff1b406d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0013b0 tx=0x7ff1bc007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:34.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.383+0000 7ff1c4ff9640 1 -- 172.21.15.64:0/3887453260 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff1c0058890 con 0x7ff1c80b6120 2024-03-07T07:19:34.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.604+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1c80b2080 con 0x7ff1c80b6120 2024-03-07T07:19:34.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.605+0000 7ff1c4ff9640 1 -- 172.21.15.64:0/3887453260 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff1c005c930 con 0x7ff1c80b6120 2024-03-07T07:19:34.605 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff1b406b4d0 msgr2=0x7ff1b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff1b406b4d0 0x7ff1b406d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0013b0 tx=0x7ff1bc007d30 comp rx=0 tx=0).stop 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c80b6120 msgr2=0x7ff1c8142cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c80b6120 0x7ff1c8142cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c7c0 tx=0x7ff1c000cc80 comp rx=0 tx=0).stop 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 shutdown_connections 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff1b406b4d0 0x7ff1b406d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c80b6120 0x7ff1c8142cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff1c80b43f0 0x7ff1c80b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 --2- 172.21.15.64:0/3887453260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff1c80ae470 0x7ff1c815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:34.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.606+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 >> 172.21.15.64:0/3887453260 conn(0x7ff1c8071550 msgr2=0x7ff1c8071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.607+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 shutdown_connections 2024-03-07T07:19:34.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:34.607+0000 7ff1ce4d4640 1 -- 172.21.15.64:0/3887453260 wait complete. 2024-03-07T07:19:34.617 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T07:19:34.617 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T07:19:34.617 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-03-07T07:19:34.617 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-07T07:19:34.617 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-07T07:19:34.617 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T07:19:34.617 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T07:19:34.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T07:19:34.618 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-07T07:19:34.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.618 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:34.633 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42702sec preferred_lft 42702sec 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T07:19:34.634 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fa:1a:1f:08:56:b8 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::48a3:bdff:feea:4c56/64 scope link 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fa:1a:1f:08:56:b8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::f81a:1fff:fe08:56b8/64 scope link tentative 2024-03-07T07:19:34.635 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:34.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.636 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T07:19:34.700 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-03-07T07:19:34.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.704 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-03-07T07:19:34.759 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-03-07T07:19:34.759 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T07:19:34.759 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T07:19:34.759 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T07:19:34.759 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T07:19:34.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.760 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:34.760 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:34.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T07:19:34.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T07:19:34.760 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T07:19:34.760 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:34.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.855 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-03-07T07:19:34.870 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T07:19:34.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.870 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-03-07T07:19:34.926 INFO:teuthology.orchestra.run.smithi064.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-07T07:19:34.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:34.927 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:34.927 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:34.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-07T07:19:34.927 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-07T07:19:34.927 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:35.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-07T07:19:35.076 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether ae:07:59:04:8e:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ac07:59ff:fe04:8e9b/64 scope link 2024-03-07T07:19:35.077 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.079 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-07T07:19:35.146 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:35.146 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:35.146 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:35.146 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.147 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:35.147 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.148 INFO:teuthology.orchestra.run.smithi064.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T07:19:35.148 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether ae:07:59:04:8e:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T07:19:35.148 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T07:19:35.148 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.148 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ac07:59ff:fe04:8e9b/64 scope link 2024-03-07T07:19:35.148 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:35.150 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-07T07:19:35.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.151 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:35.151 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:35.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-07T07:19:35.151 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-07T07:19:35.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-07T07:19:35.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-07T07:19:35.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-07T07:19:35.152 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:35.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.274 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:19:35.274 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:19:35.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.1 up 2024-03-07T07:19:35.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.1 master ceph-brx 2024-03-07T07:19:35.274 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:19:35.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:35.317 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-07T07:19:35.373 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-07T07:19:35.373 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:35.373 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-07T07:19:35.428 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-07T07:19:35.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:35.512 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:35.513 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:35.532 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:35.533 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:35.533 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: resolved to: "172.21.15.64:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-03-07T07:19:35.533 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: trying mount with new device syntax: 2@6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.cephfs=/ 2024-03-07T07:19:35.533 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.64:3300/172.21.15.120:3300/172.21.15.120:3301" will pass to kernel 2024-03-07T07:19:35.533 INFO:teuthology.orchestra.run.smithi064.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-07T07:19:35.533 INFO:teuthology.orchestra.run.smithi064.stdout: You just mounted a file system that supports labels which does not 2024-03-07T07:19:35.534 INFO:teuthology.orchestra.run.smithi064.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T07:19:35.534 INFO:teuthology.orchestra.run.smithi064.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T07:19:35.534 INFO:teuthology.orchestra.run.smithi064.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T07:19:35.535 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T07:19:35.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-07T07:19:35.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.562 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:35.617 INFO:teuthology.orchestra.run.smithi064.stdout:client4672 2024-03-07T07:19:35.618 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:35.618 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:35.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.673 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:35.728 INFO:teuthology.orchestra.run.smithi064.stdout:client4672 2024-03-07T07:19:35.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.728 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:35.783 INFO:teuthology.orchestra.run.smithi064.stdout:client4672 2024-03-07T07:19:35.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:35.783 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.client4672/status') 2024-03-07T07:19:35.848 INFO:teuthology.orchestra.run.smithi064.stdout:instance: client.4672 (3)172.21.15.64:0/2167755376 2024-03-07T07:19:35.849 INFO:teuthology.orchestra.run.smithi064.stdout:blocklisted: false 2024-03-07T07:19:35.849 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records in 2024-03-07T07:19:35.849 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records out 2024-03-07T07:19:35.849 INFO:teuthology.orchestra.run.smithi064.stderr:70 bytes copied, 2.0171e-05 s, 3.5 MB/s 2024-03-07T07:19:35.850 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-07T07:19:35.850 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T07:19:35.850 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-07T07:19:35.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T07:19:35.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.991+0000 7fc399c74640 1 Processor -- start 2024-03-07T07:19:35.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.991+0000 7fc399c74640 1 -- start start 2024-03-07T07:19:35.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc399c74640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc39415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:35.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc399c74640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc3940d3490 0x7fc3940d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:35.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc399c74640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc394141c60 con 0x7fc394162030 2024-03-07T07:19:35.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc399c74640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940d3e40 con 0x7fc3940d3490 2024-03-07T07:19:35.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc399c74640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940b44b0 con 0x7fc3941416d0 2024-03-07T07:19:35.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc3937fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc394162030 0x7fc394162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44866/0 (socket says 172.21.15.64:44866) 2024-03-07T07:19:35.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc3937fe640 1 -- 172.21.15.64:0/669383188 learned_addr learned my addr 172.21.15.64:0/669383188 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc393fff640 1 --2- 172.21.15.64:0/669383188 >> v2:172.21.15.120:3300/0 conn(0x7fc3940d3490 0x7fc3940d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.992+0000 7fc392ffd640 1 --2- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc39415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc3927fc640 1 -- 172.21.15.64:0/669383188 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077522987 0 0) 0x7fc394141c60 con 0x7fc394162030 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc3927fc640 1 -- 172.21.15.64:0/669383188 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc37c003520 con 0x7fc394162030 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc393fff640 1 -- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 msgr2=0x7fc39415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc393fff640 1 --2- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc39415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc393fff640 1 -- 172.21.15.64:0/669383188 >> v1:172.21.15.64:6789/0 conn(0x7fc394162030 legacy=0x7fc394162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc393fff640 1 -- 172.21.15.64:0/669383188 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39415ecd0 con 0x7fc3940d3490 2024-03-07T07:19:35.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc392ffd640 1 --2- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc39415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.993+0000 7fc393fff640 1 --2- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 0x7fc3940d3900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009d90 tx=0x7fc38c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb6a9cad9b056d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc3927fc640 1 -- 172.21.15.64:0/669383188 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c039070 con 0x7fc3940d3490 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc3927fc640 1 -- 172.21.15.64:0/669383188 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c02cc50 con 0x7fc3940d3490 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc3927fc640 1 -- 172.21.15.64:0/669383188 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c034440 con 0x7fc3940d3490 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 -- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 msgr2=0x7fc3940d3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 --2- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 0x7fc3940d3900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009d90 tx=0x7fc38c02bea0 comp rx=0 tx=0).stop 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 -- 172.21.15.64:0/669383188 shutdown_connections 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 --2- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 0x7fc3940d3900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:35.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 --2- 172.21.15.64:0/669383188 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc39415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 -- 172.21.15.64:0/669383188 >> 172.21.15.64:0/669383188 conn(0x7fc394070400 msgr2=0x7fc3940d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 -- 172.21.15.64:0/669383188 shutdown_connections 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.994+0000 7fc399c74640 1 -- 172.21.15.64:0/669383188 wait complete. 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.995+0000 7fc399c74640 1 Processor -- start 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.995+0000 7fc399c74640 1 -- start start 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc399c74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 0x7fc3941473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc399c74640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc394158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc399c74640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc394162030 0x7fc39415e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:36.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc399c74640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394162b20 con 0x7fc394162030 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc399c74640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39415f290 con 0x7fc3940d3490 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc399c74640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39415f570 con 0x7fc3941416d0 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc393fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc394162030 0x7fc39415e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc393fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc394162030 0x7fc39415e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35716/0 (socket says 172.21.15.64:35716) 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc393fff640 1 -- 172.21.15.64:0/1471183459 learned_addr learned my addr 172.21.15.64:0/1471183459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc3937fe640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 0x7fc3941473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.996+0000 7fc392ffd640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc394158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc393fff640 1 -- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 msgr2=0x7fc394158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc393fff640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc394158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc393fff640 1 -- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 msgr2=0x7fc3941473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc393fff640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 0x7fc3941473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc393fff640 1 -- 172.21.15.64:0/1471183459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394157410 con 0x7fc394162030 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc393fff640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc394162030 0x7fc39415e090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009d90 tx=0x7fc38c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:36.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc37bfff640 1 -- 172.21.15.64:0/1471183459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c039050 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc37bfff640 1 -- 172.21.15.64:0/1471183459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c045070 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3941576f0 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc37bfff640 1 -- 172.21.15.64:0/1471183459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c009450 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.997+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394157cb0 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.998+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc394062c90 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.998+0000 7fc37bfff640 1 -- 172.21.15.64:0/1471183459 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc38c03e610 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.999+0000 7fc37bfff640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc38006b4d0 0x7fc38006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.999+0000 7fc37bfff640 1 -- 172.21.15.64:0/1471183459 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc38c0aca10 con 0x7fc394162030 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.999+0000 7fc3937fe640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc38006b4d0 0x7fc38006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:36.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:35.999+0000 7fc3937fe640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc38006b4d0 0x7fc38006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc384005880 tx=0x7fc384009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:36.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.003+0000 7fc37bfff640 1 -- 172.21.15.64:0/1471183459 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc38c07b390 con 0x7fc394162030 2024-03-07T07:19:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.222+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc3940b88c0 con 0x7fc394162030 2024-03-07T07:19:36.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.223+0000 7fc37bfff640 1 -- 172.21.15.64:0/1471183459 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc38c07f430 con 0x7fc394162030 2024-03-07T07:19:36.223 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc38006b4d0 msgr2=0x7fc38006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc38006b4d0 0x7fc38006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc384005880 tx=0x7fc384009040 comp rx=0 tx=0).stop 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc394162030 msgr2=0x7fc39415e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc394162030 0x7fc39415e090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009d90 tx=0x7fc38c004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 shutdown_connections 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc38006b4d0 0x7fc38006d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc394162030 0x7fc39415e090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3941416d0 0x7fc394158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 --2- 172.21.15.64:0/1471183459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3940d3490 0x7fc3941473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.224+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 >> 172.21.15.64:0/1471183459 conn(0x7fc394070400 msgr2=0x7fc3940d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.225+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 shutdown_connections 2024-03-07T07:19:36.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:36.225+0000 7fc399c74640 1 -- 172.21.15.64:0/1471183459 wait complete. 2024-03-07T07:19:36.235 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T07:19:36.235 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T07:19:36.235 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-03-07T07:19:36.235 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-07T07:19:36.235 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-07T07:19:36.235 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T07:19:36.235 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T07:19:36.236 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T07:19:36.236 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-07T07:19:36.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.236 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-03-07T07:19:36.252 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:36.252 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:36.253 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T07:19:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T07:19:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42731sec preferred_lft 42731sec 2024-03-07T07:19:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::ec4:7aff:fe8f:478f/64 scope link 2024-03-07T07:19:36.254 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:36.254 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-07T07:19:36.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.254 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:36.255 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:36.255 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add name ceph-brx type bridge 2024-03-07T07:19:36.255 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr flush dev ceph-brx 2024-03-07T07:19:36.255 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set ceph-brx up 2024-03-07T07:19:36.255 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-07T07:19:36.255 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:36.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.510 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T07:19:36.540 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-03-07T07:19:36.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.540 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-03-07T07:19:36.611 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-03-07T07:19:36.612 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T07:19:36.612 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T07:19:36.612 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T07:19:36.612 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T07:19:36.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.612 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:36.612 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:36.612 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T07:19:36.612 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T07:19:36.612 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T07:19:36.612 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:36.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.888 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-03-07T07:19:36.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.903 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-03-07T07:19:36.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:36.957 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:36.958 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:36.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-07T07:19:36.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-07T07:19:36.958 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:37.039 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-07T07:19:37.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-07T07:19:37.040 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:37.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.210 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:37.210 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:37.210 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.0 up 2024-03-07T07:19:37.210 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.0 master ceph-brx 2024-03-07T07:19:37.211 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:37.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:37.253 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-07T07:19:37.305 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-07T07:19:37.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:37.306 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-07T07:19:37.360 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-07T07:19:37.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.361 DEBUG:teuthology.orchestra.run.smithi120:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:37.444 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:37.445 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:37.551 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.64:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: trying mount with new device syntax: 1@6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.cephfs=/ 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.64:3300/172.21.15.120:3300/172.21.15.120:3301" will pass to kernel 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout: You just mounted a file system that supports labels which does not 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T07:19:37.552 INFO:teuthology.orchestra.run.smithi120.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T07:19:37.553 INFO:teuthology.orchestra.run.smithi120.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T07:19:37.554 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T07:19:37.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.554 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-07T07:19:37.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.626 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:37.682 INFO:teuthology.orchestra.run.smithi120.stdout:client4681 2024-03-07T07:19:37.682 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:37.682 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:37.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.737 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:37.794 INFO:teuthology.orchestra.run.smithi120.stdout:client4681 2024-03-07T07:19:37.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.795 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:37.851 INFO:teuthology.orchestra.run.smithi120.stdout:client4681 2024-03-07T07:19:37.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:37.852 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.client4681/status') 2024-03-07T07:19:37.917 INFO:teuthology.orchestra.run.smithi120.stdout:instance: client.4681 (3)192.168.0.1:0/1616047948 2024-03-07T07:19:37.917 INFO:teuthology.orchestra.run.smithi120.stdout:blocklisted: false 2024-03-07T07:19:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records in 2024-03-07T07:19:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records out 2024-03-07T07:19:37.918 INFO:teuthology.orchestra.run.smithi120.stderr:69 bytes copied, 2.1679e-05 s, 3.2 MB/s 2024-03-07T07:19:37.919 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-07T07:19:37.919 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T07:19:37.919 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-07T07:19:37.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T07:19:38.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.032+0000 7f0146589640 1 Processor -- start 2024-03-07T07:19:38.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.032+0000 7f0146589640 1 -- start start 2024-03-07T07:19:38.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.033+0000 7f0146589640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01400d10a0 0x7f014015e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:38.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.033+0000 7f0146589640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0140071c50 con 0x7f0140072cf0 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.033+0000 7f0146589640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0140071f30 con 0x7f014015f170 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.033+0000 7f0146589640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014006c3a0 con 0x7f01400d10a0 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.033+0000 7f0144b26640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0140072cf0 0x7f01401470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44882/0 (socket says 172.21.15.64:44882) 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.033+0000 7f0144b26640 1 -- 172.21.15.64:0/2556675360 learned_addr learned my addr 172.21.15.64:0/2556675360 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.033+0000 7f013f7fe640 1 --2- 172.21.15.64:0/2556675360 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01400d10a0 0x7f014015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.034+0000 7f013effd640 1 -- 172.21.15.64:0/2556675360 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869655431 0 0) 0x7f0140071c50 con 0x7f0140072cf0 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.034+0000 7f013effd640 1 -- 172.21.15.64:0/2556675360 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0128003520 con 0x7f0140072cf0 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.034+0000 7f013effd640 1 -- 172.21.15.64:0/2556675360 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584450824 0 0) 0x7f0140071f30 con 0x7f014015f170 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.034+0000 7f013effd640 1 -- 172.21.15.64:0/2556675360 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0140071c50 con 0x7f014015f170 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.034+0000 7f013f7fe640 1 -- 172.21.15.64:0/2556675360 >> v1:172.21.15.120:6789/0 conn(0x7f014015f170 legacy=0x7f01400d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.034+0000 7f013f7fe640 1 -- 172.21.15.64:0/2556675360 >> v1:172.21.15.64:6789/0 conn(0x7f0140072cf0 legacy=0x7f01401470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:38.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.034+0000 7f013f7fe640 1 -- 172.21.15.64:0/2556675360 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140072210 con 0x7f01400d10a0 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f013f7fe640 1 --2- 172.21.15.64:0/2556675360 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01400d10a0 0x7f014015e650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01300097d0 tx=0x7f013002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f461d29ab32b6f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f013effd640 1 -- 172.21.15.64:0/2556675360 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130035020 con 0x7f01400d10a0 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f013effd640 1 -- 172.21.15.64:0/2556675360 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0130031890 con 0x7f01400d10a0 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f013effd640 1 -- 172.21.15.64:0/2556675360 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130031c40 con 0x7f01400d10a0 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f0146589640 1 -- 172.21.15.64:0/2556675360 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01400d10a0 msgr2=0x7f014015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f0146589640 1 --2- 172.21.15.64:0/2556675360 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01400d10a0 0x7f014015e650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01300097d0 tx=0x7f013002b930 comp rx=0 tx=0).stop 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f0146589640 1 -- 172.21.15.64:0/2556675360 shutdown_connections 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f0146589640 1 --2- 172.21.15.64:0/2556675360 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01400d10a0 0x7f014015e650 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f0146589640 1 -- 172.21.15.64:0/2556675360 >> 172.21.15.64:0/2556675360 conn(0x7f014006b480 msgr2=0x7f014006b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f0146589640 1 -- 172.21.15.64:0/2556675360 shutdown_connections 2024-03-07T07:19:38.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.035+0000 7f0146589640 1 -- 172.21.15.64:0/2556675360 wait complete. 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.036+0000 7f0146589640 1 Processor -- start 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.037+0000 7f0146589640 1 -- start start 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.037+0000 7f0146589640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0140072cf0 0x7f014013adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.037+0000 7f0146589640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01400d10a0 0x7f014013df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.037+0000 7f0146589640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f014015f170 0x7f014015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f0146589640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0140160560 con 0x7f0140072cf0 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f0146589640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01400727d0 con 0x7f01400d10a0 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f0146589640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0140071550 con 0x7f014015f170 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0140072cf0 0x7f014013adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0140072cf0 0x7f014013adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35720/0 (socket says 172.21.15.64:35720) 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 -- 172.21.15.64:0/3976420986 learned_addr learned my addr 172.21.15.64:0/3976420986 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013f7fe640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01400d10a0 0x7f014013df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:38.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f0144b26640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f014015f170 0x7f014015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 -- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f014015f170 msgr2=0x7f014015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f014015f170 0x7f014015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 -- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01400d10a0 msgr2=0x7f014013df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01400d10a0 0x7f014013df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 -- 172.21.15.64:0/3976420986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01400cdf80 con 0x7f0140072cf0 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013ffff640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0140072cf0 0x7f014013adb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01340010e0 tx=0x7f013400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013cff9640 1 -- 172.21.15.64:0/3976420986 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134017070 con 0x7f0140072cf0 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013cff9640 1 -- 172.21.15.64:0/3976420986 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01340052f0 con 0x7f0140072cf0 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f013cff9640 1 -- 172.21.15.64:0/3976420986 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134003830 con 0x7f0140072cf0 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014013e490 con 0x7f0140072cf0 2024-03-07T07:19:38.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.038+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01400ce500 con 0x7f0140072cf0 2024-03-07T07:19:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.039+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0140062c90 con 0x7f0140072cf0 2024-03-07T07:19:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.040+0000 7f013cff9640 1 -- 172.21.15.64:0/3976420986 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f013401a070 con 0x7f0140072cf0 2024-03-07T07:19:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.040+0000 7f013cff9640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f012c06b500 0x7f012c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.040+0000 7f013cff9640 1 -- 172.21.15.64:0/3976420986 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01340894f0 con 0x7f0140072cf0 2024-03-07T07:19:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.040+0000 7f013f7fe640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f012c06b500 0x7f012c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:38.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.041+0000 7f013f7fe640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f012c06b500 0x7f012c06d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0130036000 tx=0x7f0130008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:38.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.045+0000 7f013cff9640 1 -- 172.21.15.64:0/3976420986 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0134057d20 con 0x7f0140072cf0 2024-03-07T07:19:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.267+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01400ceea0 con 0x7f0140072cf0 2024-03-07T07:19:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.267+0000 7f013cff9640 1 -- 172.21.15.64:0/3976420986 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f013405bdc0 con 0x7f0140072cf0 2024-03-07T07:19:38.267 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f012c06b500 msgr2=0x7f012c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f012c06b500 0x7f012c06d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0130036000 tx=0x7f0130008030 comp rx=0 tx=0).stop 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0140072cf0 msgr2=0x7f014013adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0140072cf0 0x7f014013adb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01340010e0 tx=0x7f013400c890 comp rx=0 tx=0).stop 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 shutdown_connections 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f012c06b500 0x7f012c06d9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f014015f170 0x7f014015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01400d10a0 0x7f014013df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 --2- 172.21.15.64:0/3976420986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0140072cf0 0x7f014013adb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 >> 172.21.15.64:0/3976420986 conn(0x7f014006b480 msgr2=0x7f014015f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 shutdown_connections 2024-03-07T07:19:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:38.269+0000 7f0146589640 1 -- 172.21.15.64:0/3976420986 wait complete. 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T07:19:38.280 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-07T07:19:38.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.281 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-03-07T07:19:38.296 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:38.296 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:38.296 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T07:19:38.297 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42729sec preferred_lft 42729sec 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::ec4:7aff:fe8f:478f/64 scope link 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 42:b9:bd:d7:33:b2 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::ccf2:edff:fe39:6814/64 scope link 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 42:b9:bd:d7:33:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::40b9:bdff:fed7:33b2/64 scope link tentative 2024-03-07T07:19:38.298 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.299 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T07:19:38.362 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-03-07T07:19:38.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.364 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-03-07T07:19:38.419 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-03-07T07:19:38.419 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T07:19:38.419 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T07:19:38.419 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T07:19:38.419 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T07:19:38.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.420 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:38.420 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:38.420 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T07:19:38.420 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T07:19:38.420 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T07:19:38.420 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:38.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.514 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-03-07T07:19:38.528 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-07T07:19:38.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.529 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-03-07T07:19:38.584 INFO:teuthology.orchestra.run.smithi120.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-07T07:19:38.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.585 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:38.585 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:38.585 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-07T07:19:38.585 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-07T07:19:38.585 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:38.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.666 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-07T07:19:38.732 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether da:51:b4:ae:ed:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::d851:b4ff:feae:edf5/64 scope link tentative 2024-03-07T07:19:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.734 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T07:19:38.802 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether da:51:b4:ae:ed:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T07:19:38.803 INFO:teuthology.orchestra.run.smithi120.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T07:19:38.803 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.803 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::d851:b4ff:feae:edf5/64 scope link 2024-03-07T07:19:38.803 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:19:38.804 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-07T07:19:38.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.804 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:38.804 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:38.804 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-07T07:19:38.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-07T07:19:38.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-07T07:19:38.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-07T07:19:38.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-07T07:19:38.805 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:38.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:38.930 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec bash -c ' 2024-03-07T07:19:38.930 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-03-07T07:19:38.930 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.1 up 2024-03-07T07:19:38.930 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.1 master ceph-brx 2024-03-07T07:19:38.930 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-03-07T07:19:38.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:38.971 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-07T07:19:39.026 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-07T07:19:39.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:19:39.027 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-07T07:19:39.081 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-07T07:19:39.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:39.082 DEBUG:teuthology.orchestra.run.smithi120:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:39.165 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:39.165 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:39.173 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:19:39.173 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:19:39.173 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.64:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-03-07T07:19:39.174 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: trying mount with new device syntax: 3@6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.cephfs=/ 2024-03-07T07:19:39.174 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.64:3300/172.21.15.120:3300/172.21.15.120:3301" will pass to kernel 2024-03-07T07:19:39.174 INFO:teuthology.orchestra.run.smithi120.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-07T07:19:39.174 INFO:teuthology.orchestra.run.smithi120.stdout: You just mounted a file system that supports labels which does not 2024-03-07T07:19:39.174 INFO:teuthology.orchestra.run.smithi120.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T07:19:39.174 INFO:teuthology.orchestra.run.smithi120.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T07:19:39.174 INFO:teuthology.orchestra.run.smithi120.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T07:19:39.175 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T07:19:39.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:39.175 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-07T07:19:39.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:39.243 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:39.299 INFO:teuthology.orchestra.run.smithi120.stdout:client4683 2024-03-07T07:19:39.299 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:39.300 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:39.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:39.356 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:39.414 INFO:teuthology.orchestra.run.smithi120.stdout:client4683 2024-03-07T07:19:39.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:39.415 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:19:39.471 INFO:teuthology.orchestra.run.smithi120.stdout:client4683 2024-03-07T07:19:39.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:39.471 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.client4683/status') 2024-03-07T07:19:39.541 INFO:teuthology.orchestra.run.smithi120.stdout:instance: client.4683 (3)172.21.15.120:0/2122285876 2024-03-07T07:19:39.541 INFO:teuthology.orchestra.run.smithi120.stdout:blocklisted: false 2024-03-07T07:19:39.541 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records in 2024-03-07T07:19:39.541 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records out 2024-03-07T07:19:39.541 INFO:teuthology.orchestra.run.smithi120.stderr:71 bytes copied, 2.0936e-05 s, 3.4 MB/s 2024-03-07T07:19:39.542 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-07T07:19:39.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.671+0000 7fabc651d640 1 Processor -- start 2024-03-07T07:19:39.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.671+0000 7fabc651d640 1 -- start start 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc651d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc015c210 0x7fabc0072b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc651d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fabc0071c30 0x7fabc015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc651d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fabc00a9430 0x7fabc00a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc651d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc00cf1d0 con 0x7fabc00a9430 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc651d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc00ce2e0 con 0x7fabc0071c30 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc651d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00ce5c0 con 0x7fabc015c210 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fabc00a9430 0x7fabc00a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fabc00a9430 0x7fabc00a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35724/0 (socket says 172.21.15.64:35724) 2024-03-07T07:19:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 -- 172.21.15.64:0/962728505 learned_addr learned my addr 172.21.15.64:0/962728505 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabbffff640 1 --2- 172.21.15.64:0/962728505 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc015c210 0x7fabc0072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabbf7fe640 1 --2- 172.21.15.64:0/962728505 >> v2:172.21.15.120:3300/0 conn(0x7fabc0071c30 0x7fabc015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 -- 172.21.15.64:0/962728505 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc015c210 msgr2=0x7fabc0072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 --2- 172.21.15.64:0/962728505 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc015c210 0x7fabc0072b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 -- 172.21.15.64:0/962728505 >> v2:172.21.15.120:3300/0 conn(0x7fabc0071c30 msgr2=0x7fabc015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 --2- 172.21.15.64:0/962728505 >> v2:172.21.15.120:3300/0 conn(0x7fabc0071c30 0x7fabc015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.672+0000 7fabc4aba640 1 -- 172.21.15.64:0/962728505 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00ce8a0 con 0x7fabc00a9430 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.673+0000 7fabc4aba640 1 --2- 172.21.15.64:0/962728505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc00a9430 0x7fabc00a98c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c1d932683109cc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.673+0000 7fabbeffd640 1 -- 172.21.15.64:0/962728505 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb802c9e0 con 0x7fabc00a9430 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.673+0000 7fabbeffd640 1 -- 172.21.15.64:0/962728505 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb802cb80 con 0x7fabc00a9430 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.673+0000 7fabbeffd640 1 -- 172.21.15.64:0/962728505 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb80343f0 con 0x7fabc00a9430 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 -- 172.21.15.64:0/962728505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc00a9430 msgr2=0x7fabc00a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 --2- 172.21.15.64:0/962728505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc00a9430 0x7fabc00a98c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb802be40 comp rx=0 tx=0).stop 2024-03-07T07:19:39.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 -- 172.21.15.64:0/962728505 shutdown_connections 2024-03-07T07:19:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 --2- 172.21.15.64:0/962728505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc00a9430 0x7fabc00a98c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 --2- 172.21.15.64:0/962728505 >> v2:172.21.15.120:3300/0 conn(0x7fabc0071c30 0x7fabc015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 --2- 172.21.15.64:0/962728505 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc015c210 0x7fabc0072b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 -- 172.21.15.64:0/962728505 >> 172.21.15.64:0/962728505 conn(0x7fabc00b21a0 msgr2=0x7fabc00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 -- 172.21.15.64:0/962728505 shutdown_connections 2024-03-07T07:19:39.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.674+0000 7fabc651d640 1 -- 172.21.15.64:0/962728505 wait complete. 2024-03-07T07:19:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.675+0000 7fabc651d640 1 Processor -- start 2024-03-07T07:19:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.675+0000 7fabc651d640 1 -- start start 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc651d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc0071c30 0x7fabc01e2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc651d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabc00a9430 0x7fabc01f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc651d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc015c210 0x7fabc00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc651d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc015cb30 con 0x7fabc015c210 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc651d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00cee60 con 0x7fabc00a9430 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc651d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc015ccb0 con 0x7fabc0071c30 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc4aba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc015c210 0x7fabc00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc4aba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc015c210 0x7fabc00b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35730/0 (socket says 172.21.15.64:35730) 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabc4aba640 1 -- 172.21.15.64:0/1696929390 learned_addr learned my addr 172.21.15.64:0/1696929390 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabbf7fe640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabc00a9430 0x7fabc01f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.676+0000 7fabbffff640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc0071c30 0x7fabc01e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc4aba640 1 -- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc0071c30 msgr2=0x7fabc01e2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc4aba640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc0071c30 0x7fabc01e2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc4aba640 1 -- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabc00a9430 msgr2=0x7fabc01f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc4aba640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabc00a9430 0x7fabc01f14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc4aba640 1 -- 172.21.15.64:0/1696929390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc01f6270 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabbf7fe640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabc00a9430 0x7fabc01f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc4aba640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc015c210 0x7fabc00b1350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabbcff9640 1 -- 172.21.15.64:0/1696929390 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8043020 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc01f5f90 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabbcff9640 1 -- 172.21.15.64:0/1696929390 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb803e450 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabbcff9640 1 -- 172.21.15.64:0/1696929390 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8009510 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.677+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc01e1f90 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.678+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc0062c90 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.678+0000 7fabbcff9640 1 -- 172.21.15.64:0/1696929390 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fabb8046020 con 0x7fabc015c210 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.679+0000 7fabbcff9640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fabac06b500 0x7fabac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.679+0000 7fabbcff9640 1 -- 172.21.15.64:0/1696929390 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabb80ac740 con 0x7fabc015c210 2024-03-07T07:19:39.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.679+0000 7fabbffff640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fabac06b500 0x7fabac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.679+0000 7fabbffff640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fabac06b500 0x7fabac06d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009b40 tx=0x7fabb4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:39.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.683+0000 7fabbcff9640 1 -- 172.21.15.64:0/1696929390 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fabb804d030 con 0x7fabc015c210 2024-03-07T07:19:39.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.864+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabc00cc920 con 0x7fabc015c210 2024-03-07T07:19:39.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.865+0000 7fabbcff9640 1 -- 172.21.15.64:0/1696929390 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fabb807af80 con 0x7fabc015c210 2024-03-07T07:19:39.866 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:39.866 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.867+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fabac06b500 msgr2=0x7fabac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.867+0000 7fabc651d640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fabac06b500 0x7fabac06d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009b40 tx=0x7fabb4009190 comp rx=0 tx=0).stop 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.867+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc015c210 msgr2=0x7fabc00b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.867+0000 7fabc651d640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc015c210 0x7fabc00b1350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb8004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 shutdown_connections 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fabac06b500 0x7fabac06d9c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabc015c210 0x7fabc00b1350 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabc00a9430 0x7fabc01f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 --2- 172.21.15.64:0/1696929390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabc0071c30 0x7fabc01e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 >> 172.21.15.64:0/1696929390 conn(0x7fabc00b21a0 msgr2=0x7fabc0158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:39.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 shutdown_connections 2024-03-07T07:19:39.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.868+0000 7fabc651d640 1 -- 172.21.15.64:0/1696929390 wait complete. 2024-03-07T07:19:39.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:39.879 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:39.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.984+0000 7f9e35abf640 1 Processor -- start 2024-03-07T07:19:39.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.984+0000 7f9e35abf640 1 -- start start 2024-03-07T07:19:39.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e35abf640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e30145670 0x7f9e300cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e35abf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3006e150 con 0x7f9e3015cf90 2024-03-07T07:19:39.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e35abf640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3006e430 con 0x7f9e300cdbf0 2024-03-07T07:19:39.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e35abf640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3006ed60 con 0x7f9e30145670 2024-03-07T07:19:39.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e2f7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9e30145670 0x7f9e300cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e2effd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9e3015cf90 0x7f9e300cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44886/0 (socket says 172.21.15.64:44886) 2024-03-07T07:19:39.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e2effd640 1 -- 172.21.15.64:0/2535191889 learned_addr learned my addr 172.21.15.64:0/2535191889 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e2f7fe640 1 -- 172.21.15.64:0/2535191889 >> v1:172.21.15.120:6789/0 conn(0x7f9e300cdbf0 legacy=0x7f9e300719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e2f7fe640 1 -- 172.21.15.64:0/2535191889 >> v1:172.21.15.64:6789/0 conn(0x7f9e3015cf90 legacy=0x7f9e300cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e2e7fc640 1 -- 172.21.15.64:0/2535191889 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493782157 0 0) 0x7f9e3006e150 con 0x7f9e3015cf90 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.985+0000 7f9e2f7fe640 1 -- 172.21.15.64:0/2535191889 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3006e710 con 0x7f9e30145670 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e2f7fe640 1 --2- 172.21.15.64:0/2535191889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e30145670 0x7f9e300cf610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e200097d0 tx=0x7f9e2002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce5035d4aed99e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e2e7fc640 1 -- 172.21.15.64:0/2535191889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20035020 con 0x7f9e30145670 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e2e7fc640 1 -- 172.21.15.64:0/2535191889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e20031830 con 0x7f9e30145670 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e2e7fc640 1 -- 172.21.15.64:0/2535191889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20031c00 con 0x7f9e30145670 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e35abf640 1 -- 172.21.15.64:0/2535191889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e30145670 msgr2=0x7f9e300cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e35abf640 1 --2- 172.21.15.64:0/2535191889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e30145670 0x7f9e300cf610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e200097d0 tx=0x7f9e2002b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e35abf640 1 -- 172.21.15.64:0/2535191889 shutdown_connections 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e35abf640 1 --2- 172.21.15.64:0/2535191889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e30145670 0x7f9e300cf610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e35abf640 1 -- 172.21.15.64:0/2535191889 >> 172.21.15.64:0/2535191889 conn(0x7f9e300b1d30 msgr2=0x7f9e300b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:39.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.986+0000 7f9e35abf640 1 -- 172.21.15.64:0/2535191889 shutdown_connections 2024-03-07T07:19:39.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.987+0000 7f9e35abf640 1 -- 172.21.15.64:0/2535191889 wait complete. 2024-03-07T07:19:39.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 Processor -- start 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 -- start start 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300cdbf0 0x7f9e300b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e30145670 0x7f9e301f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e3015cf90 0x7f9e3013f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e30145a50 con 0x7f9e300cdbf0 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3015d420 con 0x7f9e30145670 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.988+0000 7f9e35abf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e30145bd0 con 0x7f9e3015cf90 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300cdbf0 0x7f9e300b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300cdbf0 0x7f9e300b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35748/0 (socket says 172.21.15.64:35748) 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 -- 172.21.15.64:0/1198353205 learned_addr learned my addr 172.21.15.64:0/1198353205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2effd640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e30145670 0x7f9e301f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2ffff640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e3015cf90 0x7f9e3013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 -- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e3015cf90 msgr2=0x7f9e3013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e3015cf90 0x7f9e3013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 -- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e30145670 msgr2=0x7f9e301f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e30145670 0x7f9e301f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 -- 172.21.15.64:0/1198353205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e300b36c0 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e2f7fe640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300cdbf0 0x7f9e300b2f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e20034040 tx=0x7f9e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e17fff640 1 -- 172.21.15.64:0/1198353205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20041070 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e17fff640 1 -- 172.21.15.64:0/1198353205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e20004230 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e300b39a0 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e17fff640 1 -- 172.21.15.64:0/1198353205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20035040 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.989+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3013f220 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.990+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e300b3ec0 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.991+0000 7f9e17fff640 1 -- 172.21.15.64:0/1198353205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9e20038020 con 0x7f9e300cdbf0 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.991+0000 7f9e17fff640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:39.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.991+0000 7f9e17fff640 1 -- 172.21.15.64:0/1198353205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e200abc00 con 0x7f9e300cdbf0 2024-03-07T07:19:39.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.991+0000 7f9e2effd640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:39.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.992+0000 7f9e2effd640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e24007f70 tx=0x7f9e24007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:39.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:39.996+0000 7f9e17fff640 1 -- 172.21.15.64:0/1198353205 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e200b2080 con 0x7f9e300cdbf0 2024-03-07T07:19:40.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.175+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e30062c90 con 0x7f9e300cdbf0 2024-03-07T07:19:40.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.175+0000 7f9e17fff640 1 -- 172.21.15.64:0/1198353205 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e2007a430 con 0x7f9e300cdbf0 2024-03-07T07:19:40.221 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:40.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9e1c06b4d0 msgr2=0x7f9e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e24007f70 tx=0x7f9e24007f00 comp rx=0 tx=0).stop 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300cdbf0 msgr2=0x7f9e300b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300cdbf0 0x7f9e300b2f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e20034040 tx=0x7f9e200046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 shutdown_connections 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e3015cf90 0x7f9e3013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e30145670 0x7f9e301f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 --2- 172.21.15.64:0/1198353205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e300cdbf0 0x7f9e300b2f50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 >> 172.21.15.64:0/1198353205 conn(0x7f9e300b1d30 msgr2=0x7f9e3015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 shutdown_connections 2024-03-07T07:19:40.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.177+0000 7f9e35abf640 1 -- 172.21.15.64:0/1198353205 wait complete. 2024-03-07T07:19:40.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:40.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.331+0000 7fa2906d4640 1 Processor -- start 2024-03-07T07:19:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.331+0000 7fa2906d4640 1 -- start start 2024-03-07T07:19:40.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.331+0000 7fa2906d4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2880cea70 0x7fa28815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.331+0000 7fa2906d4640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa288070eb0 con 0x7fa28815e870 2024-03-07T07:19:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.331+0000 7fa2906d4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa288071190 con 0x7fa2880d3310 2024-03-07T07:19:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.331+0000 7fa2906d4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2880cf8a0 con 0x7fa2880cea70 2024-03-07T07:19:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.332+0000 7fa28dc6f640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa2880cea70 0x7fa28815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.332+0000 7fa28e470640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa28815e870 0x7fa2880cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:44892/0 (socket says 172.21.15.64:44892) 2024-03-07T07:19:40.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.332+0000 7fa28e470640 1 -- 172.21.15.64:0/2513754495 learned_addr learned my addr 172.21.15.64:0/2513754495 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:40.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.332+0000 7fa28dc6f640 1 -- 172.21.15.64:0/2513754495 >> v1:172.21.15.120:6789/0 conn(0x7fa2880d3310 legacy=0x7fa288167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.332+0000 7fa28dc6f640 1 -- 172.21.15.64:0/2513754495 >> v1:172.21.15.64:6789/0 conn(0x7fa28815e870 legacy=0x7fa2880cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.332+0000 7fa28d46e640 1 -- 172.21.15.64:0/2513754495 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002954811 0 0) 0x7fa288070eb0 con 0x7fa28815e870 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.332+0000 7fa28dc6f640 1 -- 172.21.15.64:0/2513754495 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288071470 con 0x7fa2880cea70 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa28dc6f640 1 --2- 172.21.15.64:0/2513754495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2880cea70 0x7fa28815ec50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa2780097a0 tx=0x7fa27802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76f1e50c2a5b44c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa28d46e640 1 -- 172.21.15.64:0/2513754495 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278034070 con 0x7fa2880cea70 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa28d46e640 1 -- 172.21.15.64:0/2513754495 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa278030610 con 0x7fa2880cea70 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa28d46e640 1 -- 172.21.15.64:0/2513754495 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2780309e0 con 0x7fa2880cea70 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa2906d4640 1 -- 172.21.15.64:0/2513754495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2880cea70 msgr2=0x7fa28815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa2906d4640 1 --2- 172.21.15.64:0/2513754495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2880cea70 0x7fa28815ec50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa2780097a0 tx=0x7fa27802b960 comp rx=0 tx=0).stop 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa2906d4640 1 -- 172.21.15.64:0/2513754495 shutdown_connections 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa2906d4640 1 --2- 172.21.15.64:0/2513754495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2880cea70 0x7fa28815ec50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa2906d4640 1 -- 172.21.15.64:0/2513754495 >> 172.21.15.64:0/2513754495 conn(0x7fa2880d11e0 msgr2=0x7fa2880d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa2906d4640 1 -- 172.21.15.64:0/2513754495 shutdown_connections 2024-03-07T07:19:40.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.333+0000 7fa2906d4640 1 -- 172.21.15.64:0/2513754495 wait complete. 2024-03-07T07:19:40.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 Processor -- start 2024-03-07T07:19:40.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 -- start start 2024-03-07T07:19:40.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2880cea70 0x7fa28815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2880d3310 0x7fa2880cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28815e870 0x7fa2880ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa288070b60 con 0x7fa28815e870 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa288071a30 con 0x7fa2880d3310 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa2906d4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28815f190 con 0x7fa2880cea70 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa28ec71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28815e870 0x7fa2880ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa28ec71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28815e870 0x7fa2880ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35766/0 (socket says 172.21.15.64:35766) 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa28ec71640 1 -- 172.21.15.64:0/3917978234 learned_addr learned my addr 172.21.15.64:0/3917978234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa28dc6f640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2880d3310 0x7fa2880cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.335+0000 7fa28e470640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2880cea70 0x7fa28815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa28ec71640 1 -- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2880cea70 msgr2=0x7fa28815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa28ec71640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2880cea70 0x7fa28815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa28ec71640 1 -- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2880d3310 msgr2=0x7fa2880cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa28ec71640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2880d3310 0x7fa2880cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa28ec71640 1 -- 172.21.15.64:0/3917978234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2881487e0 con 0x7fa28815e870 2024-03-07T07:19:40.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa28ec71640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28815e870 0x7fa2880ccb10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa28000c730 tx=0x7fa28000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa27effd640 1 -- 172.21.15.64:0/3917978234 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280003b70 con 0x7fa28815e870 2024-03-07T07:19:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa27effd640 1 -- 172.21.15.64:0/3917978234 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa280003d10 con 0x7fa28815e870 2024-03-07T07:19:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2880c7950 con 0x7fa28815e870 2024-03-07T07:19:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa27effd640 1 -- 172.21.15.64:0/3917978234 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28000aa90 con 0x7fa28815e870 2024-03-07T07:19:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.336+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2880c7f10 con 0x7fa28815e870 2024-03-07T07:19:40.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.337+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa288062c90 con 0x7fa28815e870 2024-03-07T07:19:40.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.337+0000 7fa27effd640 1 -- 172.21.15.64:0/3917978234 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa28000ac30 con 0x7fa28815e870 2024-03-07T07:19:40.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.338+0000 7fa27effd640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa26c06b4d0 0x7fa26c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.338+0000 7fa27effd640 1 -- 172.21.15.64:0/3917978234 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa280089fa0 con 0x7fa28815e870 2024-03-07T07:19:40.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.338+0000 7fa28e470640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa26c06b4d0 0x7fa26c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.338+0000 7fa28e470640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa26c06b4d0 0x7fa26c06d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa284000f60 tx=0x7fa284007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.342+0000 7fa27effd640 1 -- 172.21.15.64:0/3917978234 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa2800587d0 con 0x7fa28815e870 2024-03-07T07:19:40.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.525+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa288068180 con 0x7fa28815e870 2024-03-07T07:19:40.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.525+0000 7fa27effd640 1 -- 172.21.15.64:0/3917978234 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa28005c870 con 0x7fa28815e870 2024-03-07T07:19:40.525 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:40.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa26c06b4d0 msgr2=0x7fa26c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa26c06b4d0 0x7fa26c06d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa284000f60 tx=0x7fa284007e40 comp rx=0 tx=0).stop 2024-03-07T07:19:40.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28815e870 msgr2=0x7fa2880ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28815e870 0x7fa2880ccb10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa28000c730 tx=0x7fa28000cbf0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 shutdown_connections 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa26c06b4d0 0x7fa26c06d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa28815e870 0x7fa2880ccb10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2880d3310 0x7fa2880cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 --2- 172.21.15.64:0/3917978234 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2880cea70 0x7fa28815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 >> 172.21.15.64:0/3917978234 conn(0x7fa2880d11e0 msgr2=0x7fa2880d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 shutdown_connections 2024-03-07T07:19:40.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.527+0000 7fa2906d4640 1 -- 172.21.15.64:0/3917978234 wait complete. 2024-03-07T07:19:40.538 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:40.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.642+0000 7f4838f5d640 1 Processor -- start 2024-03-07T07:19:40.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.642+0000 7f4838f5d640 1 -- start start 2024-03-07T07:19:40.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f4838f5d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4834162110 0x7f48341624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f4838f5d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834070bc0 0x7f4834071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f4838f5d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4834162a30 con 0x7f483415e8d0 2024-03-07T07:19:40.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f4838f5d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4834071610 con 0x7f4834162110 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f4838f5d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340718f0 con 0x7f4834070bc0 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f483259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4834162110 0x7f48341624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f483259c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4834162110 0x7f48341624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59038/0 (socket says 172.21.15.64:59038) 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f483259c640 1 -- 172.21.15.64:0/2019740771 learned_addr learned my addr 172.21.15.64:0/2019740771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f4831d9b640 1 --2- 172.21.15.64:0/2019740771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834070bc0 0x7f4834071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f483259c640 1 -- 172.21.15.64:0/2019740771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834070bc0 msgr2=0x7f4834071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f483259c640 1 --2- 172.21.15.64:0/2019740771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834070bc0 0x7f4834071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.643+0000 7f483259c640 1 -- 172.21.15.64:0/2019740771 >> v1:172.21.15.120:6789/0 conn(0x7f483415e8d0 legacy=0x7f483415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.644+0000 7f483259c640 1 -- 172.21.15.64:0/2019740771 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834071bd0 con 0x7f4834162110 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.644+0000 7f483259c640 1 --2- 172.21.15.64:0/2019740771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834162110 0x7f48341624f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573a5eda956a9abb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.644+0000 7f483159a640 1 -- 172.21.15.64:0/2019740771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c034070 con 0x7f4834162110 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.644+0000 7f483159a640 1 -- 172.21.15.64:0/2019740771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c030840 con 0x7f4834162110 2024-03-07T07:19:40.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.644+0000 7f483159a640 1 -- 172.21.15.64:0/2019740771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c030c10 con 0x7f4834162110 2024-03-07T07:19:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 -- 172.21.15.64:0/2019740771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834162110 msgr2=0x7f48341624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 --2- 172.21.15.64:0/2019740771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834162110 0x7f48341624f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c02b980 comp rx=0 tx=0).stop 2024-03-07T07:19:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 -- 172.21.15.64:0/2019740771 shutdown_connections 2024-03-07T07:19:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 --2- 172.21.15.64:0/2019740771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834070bc0 0x7f4834071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 --2- 172.21.15.64:0/2019740771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834162110 0x7f48341624f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 -- 172.21.15.64:0/2019740771 >> 172.21.15.64:0/2019740771 conn(0x7f48340d0e10 msgr2=0x7f48340d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 -- 172.21.15.64:0/2019740771 shutdown_connections 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.645+0000 7f4838f5d640 1 -- 172.21.15.64:0/2019740771 wait complete. 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.646+0000 7f4838f5d640 1 Processor -- start 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.646+0000 7f4838f5d640 1 -- start start 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4838f5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834070bc0 0x7f483415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4838f5d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483415e8d0 0x7f48341e8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4838f5d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834162110 0x7f4834151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4838f5d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834162e00 con 0x7f4834070bc0 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4838f5d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340d21d0 con 0x7f483415e8d0 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4838f5d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4834162f80 con 0x7f4834162110 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834070bc0 0x7f483415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834070bc0 0x7f483415c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59052/0 (socket says 172.21.15.64:59052) 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 -- 172.21.15.64:0/1621530240 learned_addr learned my addr 172.21.15.64:0/1621530240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4831d9b640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483415e8d0 0x7f48341e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f4832d9d640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834162110 0x7f4834151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 -- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834162110 msgr2=0x7f4834151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834162110 0x7f4834151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 -- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483415e8d0 msgr2=0x7f48341e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483415e8d0 0x7f48341e8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.647+0000 7f483259c640 1 -- 172.21.15.64:0/1621530240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483415cf60 con 0x7f4834070bc0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.648+0000 7f483259c640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834070bc0 0x7f483415c680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.648+0000 7f482affd640 1 -- 172.21.15.64:0/1621530240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c034040 con 0x7f4834070bc0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.648+0000 7f482affd640 1 -- 172.21.15.64:0/1621530240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c039d90 con 0x7f4834070bc0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.648+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483415d240 con 0x7f4834070bc0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.648+0000 7f482affd640 1 -- 172.21.15.64:0/1621530240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c043610 con 0x7f4834070bc0 2024-03-07T07:19:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.648+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4834151c30 con 0x7f4834070bc0 2024-03-07T07:19:40.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.649+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4834062c90 con 0x7f4834070bc0 2024-03-07T07:19:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.649+0000 7f482affd640 1 -- 172.21.15.64:0/1621530240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f482c04a060 con 0x7f4834070bc0 2024-03-07T07:19:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.650+0000 7f482affd640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f481406b4d0 0x7f481406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.650+0000 7f482affd640 1 -- 172.21.15.64:0/1621530240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f482c0abfe0 con 0x7f4834070bc0 2024-03-07T07:19:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.650+0000 7f4831d9b640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f481406b4d0 0x7f481406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.650+0000 7f4831d9b640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f481406b4d0 0x7f481406d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f483415dc20 tx=0x7f481c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.654+0000 7f482affd640 1 -- 172.21.15.64:0/1621530240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f482c03e070 con 0x7f4834070bc0 2024-03-07T07:19:40.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.832+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48340cab80 con 0x7f4834070bc0 2024-03-07T07:19:40.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.833+0000 7f482affd640 1 -- 172.21.15.64:0/1621530240 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f482c07a8a0 con 0x7f4834070bc0 2024-03-07T07:19:40.833 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f481406b4d0 msgr2=0x7f481406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f481406b4d0 0x7f481406d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f483415dc20 tx=0x7f481c009190 comp rx=0 tx=0).stop 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834070bc0 msgr2=0x7f483415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834070bc0 0x7f483415c680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 shutdown_connections 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f481406b4d0 0x7f481406d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4834162110 0x7f4834151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f483415e8d0 0x7f48341e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 --2- 172.21.15.64:0/1621530240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4834070bc0 0x7f483415c680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 >> 172.21.15.64:0/1621530240 conn(0x7f48340d0e10 msgr2=0x7f48340b7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 shutdown_connections 2024-03-07T07:19:40.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.834+0000 7f4838f5d640 1 -- 172.21.15.64:0/1621530240 wait complete. 2024-03-07T07:19:40.845 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:40.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.946+0000 7f93e2cb6640 1 Processor -- start 2024-03-07T07:19:40.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.946+0000 7f93e2cb6640 1 -- start start 2024-03-07T07:19:40.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e2cb6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93dc158230 0x7f93dc070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e2cb6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f93dc15abc0 0x7f93dc15b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e2cb6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc06da80 0x7f93dc06de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e2cb6640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93dc070400 con 0x7f93dc158230 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e2cb6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93dc0709c0 con 0x7f93dc15abc0 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e2cb6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc06e3d0 con 0x7f93dc06da80 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e0a52640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93dc158230 0x7f93dc070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93dbfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f93dc15abc0 0x7f93dc15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e0a52640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f93dc158230 0x7f93dc070e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59060/0 (socket says 172.21.15.64:59060) 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e0a52640 1 -- 172.21.15.64:0/2566306708 learned_addr learned my addr 172.21.15.64:0/2566306708 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.947+0000 7f93e1253640 1 --2- 172.21.15.64:0/2566306708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc06da80 0x7f93dc06de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.948+0000 7f93e0a52640 1 -- 172.21.15.64:0/2566306708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc06da80 msgr2=0x7f93dc06de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.948+0000 7f93e0a52640 1 --2- 172.21.15.64:0/2566306708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc06da80 0x7f93dc06de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.948+0000 7f93e0a52640 1 -- 172.21.15.64:0/2566306708 >> v2:172.21.15.120:3300/0 conn(0x7f93dc15abc0 msgr2=0x7f93dc15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.948+0000 7f93e0a52640 1 --2- 172.21.15.64:0/2566306708 >> v2:172.21.15.120:3300/0 conn(0x7f93dc15abc0 0x7f93dc15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.948+0000 7f93e0a52640 1 -- 172.21.15.64:0/2566306708 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc15b680 con 0x7f93dc158230 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.948+0000 7f93dbfff640 1 --2- 172.21.15.64:0/2566306708 >> v2:172.21.15.120:3300/0 conn(0x7f93dc15abc0 0x7f93dc15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.948+0000 7f93e0a52640 1 --2- 172.21.15.64:0/2566306708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc158230 0x7f93dc070e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0afb30 tx=0x7f93d002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b241aad521d03c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93db7fe640 1 -- 172.21.15.64:0/2566306708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d002e070 con 0x7f93dc158230 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93db7fe640 1 -- 172.21.15.64:0/2566306708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d0037070 con 0x7f93dc158230 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93db7fe640 1 -- 172.21.15.64:0/2566306708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0031a00 con 0x7f93dc158230 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 -- 172.21.15.64:0/2566306708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc158230 msgr2=0x7f93dc070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/2566306708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc158230 0x7f93dc070e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0afb30 tx=0x7f93d002b880 comp rx=0 tx=0).stop 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 -- 172.21.15.64:0/2566306708 shutdown_connections 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/2566306708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc06da80 0x7f93dc06de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/2566306708 >> v2:172.21.15.120:3300/0 conn(0x7f93dc15abc0 0x7f93dc15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/2566306708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc158230 0x7f93dc070e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 -- 172.21.15.64:0/2566306708 >> 172.21.15.64:0/2566306708 conn(0x7f93dc072490 msgr2=0x7f93dc072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 -- 172.21.15.64:0/2566306708 shutdown_connections 2024-03-07T07:19:40.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.949+0000 7f93e2cb6640 1 -- 172.21.15.64:0/2566306708 wait complete. 2024-03-07T07:19:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.950+0000 7f93e2cb6640 1 Processor -- start 2024-03-07T07:19:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.950+0000 7f93e2cb6640 1 -- start start 2024-03-07T07:19:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e2cb6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93dc06da80 0x7f93dc144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e2cb6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc158230 0x7f93dc156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e2cb6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc15abc0 0x7f93dc157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e2cb6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc071390 con 0x7f93dc15abc0 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e2cb6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc070650 con 0x7f93dc06da80 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e2cb6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc071a90 con 0x7f93dc158230 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e1253640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc15abc0 0x7f93dc157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e1253640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc15abc0 0x7f93dc157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59070/0 (socket says 172.21.15.64:59070) 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e1253640 1 -- 172.21.15.64:0/1235182610 learned_addr learned my addr 172.21.15.64:0/1235182610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93e0a52640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93dc06da80 0x7f93dc144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.951+0000 7f93dbfff640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc158230 0x7f93dc156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e1253640 1 -- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc158230 msgr2=0x7f93dc156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e1253640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc158230 0x7f93dc156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e1253640 1 -- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93dc06da80 msgr2=0x7f93dc144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:40.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e1253640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93dc06da80 0x7f93dc144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e1253640 1 -- 172.21.15.64:0/1235182610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc148e50 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e1253640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc15abc0 0x7f93dc157440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93d400ef00 tx=0x7f93d400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93d97fa640 1 -- 172.21.15.64:0/1235182610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4018020 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93d97fa640 1 -- 172.21.15.64:0/1235182610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d400bd20 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93dc149130 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93d97fa640 1 -- 172.21.15.64:0/1235182610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4014640 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.952+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93dc14ec60 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.953+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93dc14ee60 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.953+0000 7f93d97fa640 1 -- 172.21.15.64:0/1235182610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f93d401b020 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.954+0000 7f93d97fa640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f93c006b4d0 0x7f93c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.954+0000 7f93d97fa640 1 -- 172.21.15.64:0/1235182610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93d408a680 con 0x7f93dc15abc0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.954+0000 7f93e0a52640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f93c006b4d0 0x7f93c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:40.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.954+0000 7f93e0a52640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f93c006b4d0 0x7f93c006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93d0036000 tx=0x7f93d003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:40.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:40.958+0000 7f93d97fa640 1 -- 172.21.15.64:0/1235182610 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f93dc14ee60 con 0x7f93dc15abc0 2024-03-07T07:19:41.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.144+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93dc145420 con 0x7f93dc15abc0 2024-03-07T07:19:41.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.145+0000 7f93d97fa640 1 -- 172.21.15.64:0/1235182610 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f93d4022030 con 0x7f93dc15abc0 2024-03-07T07:19:41.145 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:41.145 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f93c006b4d0 msgr2=0x7f93c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f93c006b4d0 0x7f93c006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93d0036000 tx=0x7f93d003d040 comp rx=0 tx=0).stop 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc15abc0 msgr2=0x7f93dc157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc15abc0 0x7f93dc157440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93d400ef00 tx=0x7f93d400b480 comp rx=0 tx=0).stop 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 shutdown_connections 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f93c006b4d0 0x7f93c006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93dc15abc0 0x7f93dc157440 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93dc158230 0x7f93dc156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 --2- 172.21.15.64:0/1235182610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93dc06da80 0x7f93dc144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 >> 172.21.15.64:0/1235182610 conn(0x7f93dc072490 msgr2=0x7f93dc15be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:41.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 shutdown_connections 2024-03-07T07:19:41.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.147+0000 7f93e2cb6640 1 -- 172.21.15.64:0/1235182610 wait complete. 2024-03-07T07:19:41.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:41.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.262+0000 7f6c6cbb8640 1 Processor -- start 2024-03-07T07:19:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.262+0000 7f6c6cbb8640 1 -- start start 2024-03-07T07:19:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6cbb8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c6815e870 0x7f6c680cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6cbb8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c680cea70 0x7f6c6815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6cbb8640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c68070eb0 con 0x7f6c6815f190 2024-03-07T07:19:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6cbb8640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c68071190 con 0x7f6c6815e870 2024-03-07T07:19:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6cbb8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68071470 con 0x7f6c680cea70 2024-03-07T07:19:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6659c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6c6815e870 0x7f6c680cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c66d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6c6815f190 0x7f6c680d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45946/0 (socket says 172.21.15.64:45946) 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c66d9d640 1 -- 172.21.15.64:0/1716308365 learned_addr learned my addr 172.21.15.64:0/1716308365 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c5ffff640 1 --2- 172.21.15.64:0/1716308365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c680cea70 0x7f6c6815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6659c640 1 -- 172.21.15.64:0/1716308365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c680cea70 msgr2=0x7f6c6815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6659c640 1 --2- 172.21.15.64:0/1716308365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c680cea70 0x7f6c6815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c6659c640 1 -- 172.21.15.64:0/1716308365 >> v1:172.21.15.64:6789/0 conn(0x7f6c6815f190 legacy=0x7f6c680d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.263+0000 7f6c65d9b640 1 -- 172.21.15.64:0/1716308365 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660697543 0 0) 0x7f6c68070eb0 con 0x7f6c6815f190 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.264+0000 7f6c6659c640 1 -- 172.21.15.64:0/1716308365 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68071750 con 0x7f6c6815e870 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.264+0000 7f6c6659c640 1 --2- 172.21.15.64:0/1716308365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cf190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6c580097a0 tx=0x7f6c5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2075955b8e41c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.264+0000 7f6c65d9b640 1 -- 172.21.15.64:0/1716308365 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58035020 con 0x7f6c6815e870 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.264+0000 7f6c65d9b640 1 -- 172.21.15.64:0/1716308365 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c58031830 con 0x7f6c6815e870 2024-03-07T07:19:41.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.264+0000 7f6c65d9b640 1 -- 172.21.15.64:0/1716308365 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58031c00 con 0x7f6c6815e870 2024-03-07T07:19:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/1716308365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 msgr2=0x7f6c680cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/1716308365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cf190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6c580097a0 tx=0x7f6c5802b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/1716308365 shutdown_connections 2024-03-07T07:19:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/1716308365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c680cea70 0x7f6c6815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/1716308365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cf190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/1716308365 >> 172.21.15.64:0/1716308365 conn(0x7f6c680d11e0 msgr2=0x7f6c680d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/1716308365 shutdown_connections 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.265+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/1716308365 wait complete. 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.266+0000 7f6c6cbb8640 1 Processor -- start 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.266+0000 7f6c6cbb8640 1 -- start start 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c6cbb8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c680cea70 0x7f6c6814e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c6cbb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c6cbb8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c6815f190 0x7f6c6814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c6cbb8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68070b60 con 0x7f6c6815e870 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c6cbb8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6815f600 con 0x7f6c680cea70 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c6cbb8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680cf6d0 con 0x7f6c6815f190 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59074/0 (socket says 172.21.15.64:59074) 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 -- 172.21.15.64:0/3458833899 learned_addr learned my addr 172.21.15.64:0/3458833899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c6659c640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c680cea70 0x7f6c6814e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c66d9d640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c6815f190 0x7f6c6814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 -- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c6815f190 msgr2=0x7f6c6814ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c6815f190 0x7f6c6814ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 -- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c680cea70 msgr2=0x7f6c6814e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c680cea70 0x7f6c6814e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.267+0000 7f6c5ffff640 1 -- 172.21.15.64:0/3458833899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680ce210 con 0x7f6c6815e870 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.268+0000 7f6c5ffff640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cb500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e8a0 tx=0x7f6c5000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.268+0000 7f6c5effd640 1 -- 172.21.15.64:0/3458833899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50017070 con 0x7f6c6815e870 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.268+0000 7f6c5effd640 1 -- 172.21.15.64:0/3458833899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5000c940 con 0x7f6c6815e870 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.268+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c680ce4f0 con 0x7f6c6815e870 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.268+0000 7f6c5effd640 1 -- 172.21.15.64:0/3458833899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000cd30 con 0x7f6c6815e870 2024-03-07T07:19:41.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.268+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c680ce7d0 con 0x7f6c6815e870 2024-03-07T07:19:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.269+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c68062c90 con 0x7f6c6815e870 2024-03-07T07:19:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.269+0000 7f6c5effd640 1 -- 172.21.15.64:0/3458833899 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6c5001a070 con 0x7f6c6815e870 2024-03-07T07:19:41.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.270+0000 7f6c5effd640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6c4806b4d0 0x7f6c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.270+0000 7f6c5effd640 1 -- 172.21.15.64:0/3458833899 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c5008a380 con 0x7f6c6815e870 2024-03-07T07:19:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.270+0000 7f6c6659c640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6c4806b4d0 0x7f6c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.270+0000 7f6c6659c640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6c4806b4d0 0x7f6c4806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c58036000 tx=0x7f6c58008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.274+0000 7f6c5effd640 1 -- 172.21.15.64:0/3458833899 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c50058bb0 con 0x7f6c6815e870 2024-03-07T07:19:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.456+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c68157aa0 con 0x7f6c6815e870 2024-03-07T07:19:41.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.456+0000 7f6c5effd640 1 -- 172.21.15.64:0/3458833899 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c5005cc50 con 0x7f6c6815e870 2024-03-07T07:19:41.457 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6c4806b4d0 msgr2=0x7f6c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6c4806b4d0 0x7f6c4806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c58036000 tx=0x7f6c58008030 comp rx=0 tx=0).stop 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 msgr2=0x7f6c680cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cb500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e8a0 tx=0x7f6c5000ed60 comp rx=0 tx=0).stop 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 shutdown_connections 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6c4806b4d0 0x7f6c4806d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c6815e870 0x7f6c680cb500 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6c6815f190 0x7f6c6814ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 --2- 172.21.15.64:0/3458833899 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c680cea70 0x7f6c6814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 >> 172.21.15.64:0/3458833899 conn(0x7f6c680d11e0 msgr2=0x7f6c680c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 shutdown_connections 2024-03-07T07:19:41.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.458+0000 7f6c6cbb8640 1 -- 172.21.15.64:0/3458833899 wait complete. 2024-03-07T07:19:41.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.573+0000 7f2cb05eb640 1 Processor -- start 2024-03-07T07:19:41.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.573+0000 7f2cb05eb640 1 -- start start 2024-03-07T07:19:41.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2cb05eb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2cb05eb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ca80d3310 0x7f2ca815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2cb05eb640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca8070eb0 con 0x7f2ca815e870 2024-03-07T07:19:41.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2cb05eb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca815f6d0 con 0x7f2ca80d3310 2024-03-07T07:19:41.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2cb05eb640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80cf8a0 con 0x7f2ca80cea70 2024-03-07T07:19:41.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2caeb88640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ca80d3310 0x7f2ca815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2caeb88640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2ca80d3310 0x7f2ca815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59080/0 (socket says 172.21.15.64:59080) 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2caeb88640 1 -- 172.21.15.64:0/1813827906 learned_addr learned my addr 172.21.15.64:0/1813827906 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2cae387640 1 --1- 172.21.15.64:0/1813827906 >> v1:172.21.15.120:6789/0 conn(0x7f2ca815e870 0x7f2ca80cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:39610/0 (socket says 172.21.15.64:39610) 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2cadb86640 1 --2- 172.21.15.64:0/1813827906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2caeb88640 1 -- 172.21.15.64:0/1813827906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 msgr2=0x7f2ca815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2caeb88640 1 --2- 172.21.15.64:0/1813827906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2caeb88640 1 -- 172.21.15.64:0/1813827906 >> v1:172.21.15.120:6789/0 conn(0x7f2ca815e870 legacy=0x7f2ca80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.574+0000 7f2caeb88640 1 -- 172.21.15.64:0/1813827906 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca8071190 con 0x7f2ca80d3310 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.575+0000 7f2caeb88640 1 --2- 172.21.15.64:0/1813827906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca80d3310 0x7f2ca815f190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009e50 tx=0x7f2ca002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de486b494fef63b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.575+0000 7f2cad385640 1 -- 172.21.15.64:0/1813827906 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca002cb00 con 0x7f2ca80d3310 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.575+0000 7f2cad385640 1 -- 172.21.15.64:0/1813827906 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca002cca0 con 0x7f2ca80d3310 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.575+0000 7f2cad385640 1 -- 172.21.15.64:0/1813827906 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0034610 con 0x7f2ca80d3310 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 -- 172.21.15.64:0/1813827906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca80d3310 msgr2=0x7f2ca815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/1813827906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca80d3310 0x7f2ca815f190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009e50 tx=0x7f2ca002bf60 comp rx=0 tx=0).stop 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 -- 172.21.15.64:0/1813827906 shutdown_connections 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/1813827906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca80d3310 0x7f2ca815f190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/1813827906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 -- 172.21.15.64:0/1813827906 >> 172.21.15.64:0/1813827906 conn(0x7f2ca80d11e0 msgr2=0x7f2ca80d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 -- 172.21.15.64:0/1813827906 shutdown_connections 2024-03-07T07:19:41.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.576+0000 7f2cb05eb640 1 -- 172.21.15.64:0/1813827906 wait complete. 2024-03-07T07:19:41.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.577+0000 7f2cb05eb640 1 Processor -- start 2024-03-07T07:19:41.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.577+0000 7f2cb05eb640 1 -- start start 2024-03-07T07:19:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca814f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca80d3310 0x7f2ca80b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca815e870 0x7f2ca80cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8071a30 con 0x7f2ca815e870 2024-03-07T07:19:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8071750 con 0x7f2ca80d3310 2024-03-07T07:19:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8070b60 con 0x7f2ca80cea70 2024-03-07T07:19:41.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca815e870 0x7f2ca80cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca815e870 0x7f2ca80cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59088/0 (socket says 172.21.15.64:59088) 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 -- 172.21.15.64:0/3883212644 learned_addr learned my addr 172.21.15.64:0/3883212644 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cadb86640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca80d3310 0x7f2ca80b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cae387640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca814f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 -- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 msgr2=0x7f2ca814f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca814f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 -- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca80d3310 msgr2=0x7f2ca80b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca80d3310 0x7f2ca80b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 -- 172.21.15.64:0/3883212644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80ccd30 con 0x7f2ca815e870 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2caeb88640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca815e870 0x7f2ca80cc7f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036040 tx=0x7f2ca0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2c9effd640 1 -- 172.21.15.64:0/3883212644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0043020 con 0x7f2ca815e870 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2c9effd640 1 -- 172.21.15.64:0/3883212644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca002ce50 con 0x7f2ca815e870 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2c9effd640 1 -- 172.21.15.64:0/3883212644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0009510 con 0x7f2ca815e870 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca80ccfb0 con 0x7f2ca815e870 2024-03-07T07:19:41.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.578+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca80cd570 con 0x7f2ca815e870 2024-03-07T07:19:41.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.580+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8062c90 con 0x7f2ca815e870 2024-03-07T07:19:41.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.580+0000 7f2c9effd640 1 -- 172.21.15.64:0/3883212644 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2ca0046020 con 0x7f2ca815e870 2024-03-07T07:19:41.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.580+0000 7f2c9effd640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.580+0000 7f2c9effd640 1 -- 172.21.15.64:0/3883212644 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ca00ac6d0 con 0x7f2ca815e870 2024-03-07T07:19:41.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.580+0000 7f2cae387640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.581+0000 7f2cae387640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4007fd0 tx=0x7f2ca4007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.585+0000 7f2c9effd640 1 -- 172.21.15.64:0/3883212644 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ca004d030 con 0x7f2ca815e870 2024-03-07T07:19:41.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.765+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ca80cf450 con 0x7f2ca815e870 2024-03-07T07:19:41.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.765+0000 7f2c9effd640 1 -- 172.21.15.64:0/3883212644 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ca007fe20 con 0x7f2ca815e870 2024-03-07T07:19:41.765 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:41.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c8c06b4d0 msgr2=0x7f2c8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4007fd0 tx=0x7f2ca4007e80 comp rx=0 tx=0).stop 2024-03-07T07:19:41.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca815e870 msgr2=0x7f2ca80cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca815e870 0x7f2ca80cc7f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036040 tx=0x7f2ca0004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 shutdown_connections 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ca815e870 0x7f2ca80cc7f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca80d3310 0x7f2ca80b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 --2- 172.21.15.64:0/3883212644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2ca80cea70 0x7f2ca814f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 >> 172.21.15.64:0/3883212644 conn(0x7f2ca80d11e0 msgr2=0x7f2ca80c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 shutdown_connections 2024-03-07T07:19:41.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.767+0000 7f2cb05eb640 1 -- 172.21.15.64:0/3883212644 wait complete. 2024-03-07T07:19:41.777 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.882+0000 7ff566feb640 1 Processor -- start 2024-03-07T07:19:41.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.882+0000 7ff566feb640 1 -- start start 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff566feb640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff56015e350 0x7ff560161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff566feb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff5601623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff566feb640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff560071ff0 con 0x7ff560071280 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff566feb640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5600d2160 con 0x7ff56015e350 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff566feb640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5600d23b0 con 0x7ff5600b7200 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff564d87640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff56015e350 0x7ff560161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff565588640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff560071280 0x7ff560071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45956/0 (socket says 172.21.15.64:45956) 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff565588640 1 -- 172.21.15.64:0/208644146 learned_addr learned my addr 172.21.15.64:0/208644146 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.883+0000 7ff55ffff640 1 --2- 172.21.15.64:0/208644146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff5601623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.884+0000 7ff564d87640 1 -- 172.21.15.64:0/208644146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 msgr2=0x7ff5601623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.884+0000 7ff564d87640 1 --2- 172.21.15.64:0/208644146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff5601623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.884+0000 7ff564d87640 1 -- 172.21.15.64:0/208644146 >> v1:172.21.15.64:6789/0 conn(0x7ff560071280 legacy=0x7ff560071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.884+0000 7ff564d87640 1 -- 172.21.15.64:0/208644146 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5600d2690 con 0x7ff56015e350 2024-03-07T07:19:41.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.884+0000 7ff55f7fe640 1 -- 172.21.15.64:0/208644146 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454042396 0 0) 0x7ff560071ff0 con 0x7ff560071280 2024-03-07T07:19:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.884+0000 7ff564d87640 1 --2- 172.21.15.64:0/208644146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff56015e350 0x7ff560161eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff558009d90 tx=0x7ff55802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9f3aeba12edfc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff55f7fe640 1 -- 172.21.15.64:0/208644146 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558039070 con 0x7ff56015e350 2024-03-07T07:19:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff55f7fe640 1 -- 172.21.15.64:0/208644146 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55802cc50 con 0x7ff56015e350 2024-03-07T07:19:41.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff55f7fe640 1 -- 172.21.15.64:0/208644146 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558034440 con 0x7ff56015e350 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 -- 172.21.15.64:0/208644146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff56015e350 msgr2=0x7ff560161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 --2- 172.21.15.64:0/208644146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff56015e350 0x7ff560161eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff558009d90 tx=0x7ff55802bea0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 -- 172.21.15.64:0/208644146 shutdown_connections 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 --2- 172.21.15.64:0/208644146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff5601623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 --2- 172.21.15.64:0/208644146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff56015e350 0x7ff560161eb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 -- 172.21.15.64:0/208644146 >> 172.21.15.64:0/208644146 conn(0x7ff5600d0e70 msgr2=0x7ff5600d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 -- 172.21.15.64:0/208644146 shutdown_connections 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.885+0000 7ff566feb640 1 -- 172.21.15.64:0/208644146 wait complete. 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.886+0000 7ff566feb640 1 Processor -- start 2024-03-07T07:19:41.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.886+0000 7ff566feb640 1 -- start start 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff566feb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff560071280 0x7ff5600c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff566feb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff560160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff566feb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff56015e350 0x7ff560161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff566feb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5600d2e90 con 0x7ff560071280 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff566feb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56015e730 con 0x7ff56015e350 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff566feb640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5600d3010 con 0x7ff5600b7200 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff564d87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff560071280 0x7ff5600c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff564d87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff560071280 0x7ff5600c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59108/0 (socket says 172.21.15.64:59108) 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff564d87640 1 -- 172.21.15.64:0/3401534743 learned_addr learned my addr 172.21.15.64:0/3401534743 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff55ffff640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff560160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff565588640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff56015e350 0x7ff560161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff564d87640 1 -- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 msgr2=0x7ff560160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.887+0000 7ff564d87640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff560160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff564d87640 1 -- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff56015e350 msgr2=0x7ff560161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff564d87640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff56015e350 0x7ff560161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff564d87640 1 -- 172.21.15.64:0/3401534743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5600ca010 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff564d87640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff560071280 0x7ff5600c9ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff558037040 tx=0x7ff558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff55d7fa640 1 -- 172.21.15.64:0/3401534743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558044020 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff55d7fa640 1 -- 172.21.15.64:0/3401534743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55802ca50 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff55d7fa640 1 -- 172.21.15.64:0/3401534743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558039040 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5601f4a40 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.888+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5600c7cd0 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.889+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff560062c90 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.889+0000 7ff55d7fa640 1 -- 172.21.15.64:0/3401534743 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff558047020 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.890+0000 7ff55d7fa640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff54006b4d0 0x7ff54006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.890+0000 7ff55d7fa640 1 -- 172.21.15.64:0/3401534743 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5580ac900 con 0x7ff560071280 2024-03-07T07:19:41.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.890+0000 7ff55ffff640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff54006b4d0 0x7ff54006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:41.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.890+0000 7ff55ffff640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff54006b4d0 0x7ff54006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff5601566f0 tx=0x7ff550009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:41.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:41.894+0000 7ff55d7fa640 1 -- 172.21.15.64:0/3401534743 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff55804e030 con 0x7ff560071280 2024-03-07T07:19:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.073+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5600d1d10 con 0x7ff560071280 2024-03-07T07:19:42.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.074+0000 7ff55d7fa640 1 -- 172.21.15.64:0/3401534743 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff55807b1c0 con 0x7ff560071280 2024-03-07T07:19:42.074 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff54006b4d0 msgr2=0x7ff54006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff54006b4d0 0x7ff54006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff5601566f0 tx=0x7ff550009190 comp rx=0 tx=0).stop 2024-03-07T07:19:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff560071280 msgr2=0x7ff5600c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff560071280 0x7ff5600c9ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff558037040 tx=0x7ff558004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 shutdown_connections 2024-03-07T07:19:42.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff54006b4d0 0x7ff54006d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff56015e350 0x7ff560161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5600b7200 0x7ff560160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 --2- 172.21.15.64:0/3401534743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff560071280 0x7ff5600c9ad0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 >> 172.21.15.64:0/3401534743 conn(0x7ff5600d0e70 msgr2=0x7ff5600b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 shutdown_connections 2024-03-07T07:19:42.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.076+0000 7ff566feb640 1 -- 172.21.15.64:0/3401534743 wait complete. 2024-03-07T07:19:42.086 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:42.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.187+0000 7f7589178640 1 Processor -- start 2024-03-07T07:19:42.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.187+0000 7f7589178640 1 -- start start 2024-03-07T07:19:42.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7589178640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f758413dd70 0x7f758415f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7589178640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f75840b31e0 0x7f75840cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7589178640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758413e150 con 0x7f758415ea40 2024-03-07T07:19:42.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7589178640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758406f000 con 0x7f75840b31e0 2024-03-07T07:19:42.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7589178640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758406e3f0 con 0x7f758413dd70 2024-03-07T07:19:42.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f758413dd70 0x7f758415f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f758359e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f75840b31e0 0x7f75840cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f758413dd70 0x7f758415f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:43784/0 (socket says 172.21.15.64:43784) 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f758359e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f75840b31e0 0x7f75840cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:41310/0 (socket says 172.21.15.64:41310) 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f7582d9d640 1 -- 172.21.15.64:0/532945934 learned_addr learned my addr 172.21.15.64:0/532945934 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.188+0000 7f758259c640 1 --1- 172.21.15.64:0/532945934 >> v1:172.21.15.120:6789/0 conn(0x7f758415ea40 0x7f75840b2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:39616/0 (socket says 172.21.15.64:39616) 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7582d9d640 1 -- 172.21.15.64:0/532945934 >> v2:172.21.15.120:3300/0 conn(0x7f75840b31e0 msgr2=0x7f75840cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7582d9d640 1 --2- 172.21.15.64:0/532945934 >> v2:172.21.15.120:3300/0 conn(0x7f75840b31e0 0x7f75840cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7582d9d640 1 -- 172.21.15.64:0/532945934 >> v1:172.21.15.120:6789/0 conn(0x7f758415ea40 legacy=0x7f75840b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7582d9d640 1 -- 172.21.15.64:0/532945934 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758406e6d0 con 0x7f758413dd70 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7582d9d640 1 --2- 172.21.15.64:0/532945934 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f758413dd70 0x7f758415f270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f757400d150 tx=0x7f757402e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce4a8b75c70116d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7581d9b640 1 -- 172.21.15.64:0/532945934 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757400b0d0 con 0x7f758413dd70 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7581d9b640 1 -- 172.21.15.64:0/532945934 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400b270 con 0x7f758413dd70 2024-03-07T07:19:42.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.189+0000 7f7581d9b640 1 -- 172.21.15.64:0/532945934 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574009600 con 0x7f758413dd70 2024-03-07T07:19:42.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 -- 172.21.15.64:0/532945934 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f758413dd70 msgr2=0x7f758415f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 --2- 172.21.15.64:0/532945934 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f758413dd70 0x7f758415f270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f757400d150 tx=0x7f757402e160 comp rx=0 tx=0).stop 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 -- 172.21.15.64:0/532945934 shutdown_connections 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 --2- 172.21.15.64:0/532945934 >> v2:172.21.15.120:3300/0 conn(0x7f75840b31e0 0x7f75840cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 --2- 172.21.15.64:0/532945934 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f758413dd70 0x7f758415f270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 -- 172.21.15.64:0/532945934 >> 172.21.15.64:0/532945934 conn(0x7f758406b040 msgr2=0x7f758406b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 -- 172.21.15.64:0/532945934 shutdown_connections 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.190+0000 7f7589178640 1 -- 172.21.15.64:0/532945934 wait complete. 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.191+0000 7f7589178640 1 Processor -- start 2024-03-07T07:19:42.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.191+0000 7f7589178640 1 -- start start 2024-03-07T07:19:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f7589178640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75840b31e0 0x7f7584150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f7589178640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758413dd70 0x7f758414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f7589178640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f758415ea40 0x7f75840c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f7589178640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758415f7b0 con 0x7f758415ea40 2024-03-07T07:19:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f7589178640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758406ec90 con 0x7f758413dd70 2024-03-07T07:19:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f7589178640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840cfd20 con 0x7f75840b31e0 2024-03-07T07:19:42.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f758359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f758415ea40 0x7f75840c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f758359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f758415ea40 0x7f75840c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59112/0 (socket says 172.21.15.64:59112) 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f758359e640 1 -- 172.21.15.64:0/3423364682 learned_addr learned my addr 172.21.15.64:0/3423364682 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f7582d9d640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75840b31e0 0x7f7584150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.192+0000 7f758259c640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758413dd70 0x7f758414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7582d9d640 1 -- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758413dd70 msgr2=0x7f758414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7582d9d640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758413dd70 0x7f758414ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7582d9d640 1 -- 172.21.15.64:0/3423364682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f758415ea40 msgr2=0x7f75840c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7582d9d640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f758415ea40 0x7f75840c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7582d9d640 1 -- 172.21.15.64:0/3423364682 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584151bb0 con 0x7f75840b31e0 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7582d9d640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75840b31e0 0x7f7584150480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f7574004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f757b7fe640 1 -- 172.21.15.64:0/3423364682 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574045020 con 0x7f75840b31e0 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f757b7fe640 1 -- 172.21.15.64:0/3423364682 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400b420 con 0x7f75840b31e0 2024-03-07T07:19:42.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7584150b10 con 0x7f75840b31e0 2024-03-07T07:19:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f757b7fe640 1 -- 172.21.15.64:0/3423364682 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757400b0d0 con 0x7f75840b31e0 2024-03-07T07:19:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.193+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75841510d0 con 0x7f75840b31e0 2024-03-07T07:19:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.194+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7584062c90 con 0x7f75840b31e0 2024-03-07T07:19:42.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.194+0000 7f757b7fe640 1 -- 172.21.15.64:0/3423364682 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7574048020 con 0x7f75840b31e0 2024-03-07T07:19:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.195+0000 7f757b7fe640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f756406b4d0 0x7f756406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.195+0000 7f757b7fe640 1 -- 172.21.15.64:0/3423364682 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75740aeaf0 con 0x7f75840b31e0 2024-03-07T07:19:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.195+0000 7f758259c640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f756406b4d0 0x7f756406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.195+0000 7f758259c640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f756406b4d0 0x7f756406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f75841e6350 tx=0x7f756c007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.204+0000 7f757b7fe640 1 -- 172.21.15.64:0/3423364682 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f757404f030 con 0x7f75840b31e0 2024-03-07T07:19:42.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.386+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75840ceab0 con 0x7f75840b31e0 2024-03-07T07:19:42.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.387+0000 7f757b7fe640 1 -- 172.21.15.64:0/3423364682 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f757407d320 con 0x7f75840b31e0 2024-03-07T07:19:42.388 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:42.388 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f756406b4d0 msgr2=0x7f756406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f756406b4d0 0x7f756406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f75841e6350 tx=0x7f756c007e80 comp rx=0 tx=0).stop 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75840b31e0 msgr2=0x7f7584150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75840b31e0 0x7f7584150480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f7574004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 shutdown_connections 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f756406b4d0 0x7f756406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f758415ea40 0x7f75840c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f758413dd70 0x7f758414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 --2- 172.21.15.64:0/3423364682 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75840b31e0 0x7f7584150480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 >> 172.21.15.64:0/3423364682 conn(0x7f758406b040 msgr2=0x7f758413c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:42.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 shutdown_connections 2024-03-07T07:19:42.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.389+0000 7f7589178640 1 -- 172.21.15.64:0/3423364682 wait complete. 2024-03-07T07:19:42.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:42.401 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:42.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.505+0000 7fa166497640 1 Processor -- start 2024-03-07T07:19:42.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.505+0000 7fa166497640 1 -- start start 2024-03-07T07:19:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa166497640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa16015e870 0x7fa1600cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa166497640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa160070eb0 con 0x7fa1600d3310 2024-03-07T07:19:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa166497640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa160071190 con 0x7fa1600cea70 2024-03-07T07:19:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa166497640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa160071470 con 0x7fa16015e870 2024-03-07T07:19:42.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa15ffff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa16015e870 0x7fa1600cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa164a34640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa1600d3310 0x7fa160167e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45960/0 (socket says 172.21.15.64:45960) 2024-03-07T07:19:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa164a34640 1 -- 172.21.15.64:0/487220443 learned_addr learned my addr 172.21.15.64:0/487220443 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa15ffff640 1 -- 172.21.15.64:0/487220443 >> v1:172.21.15.120:6789/0 conn(0x7fa1600cea70 legacy=0x7fa16015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa15ffff640 1 -- 172.21.15.64:0/487220443 >> v1:172.21.15.64:6789/0 conn(0x7fa1600d3310 legacy=0x7fa160167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa15ffff640 1 -- 172.21.15.64:0/487220443 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160071750 con 0x7fa16015e870 2024-03-07T07:19:42.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.506+0000 7fa15effd640 1 -- 172.21.15.64:0/487220443 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820450049 0 0) 0x7fa160070eb0 con 0x7fa1600d3310 2024-03-07T07:19:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.507+0000 7fa15ffff640 1 --2- 172.21.15.64:0/487220443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600cf190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1540097a0 tx=0x7fa15402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72b80d6b5840c3a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.507+0000 7fa15effd640 1 -- 172.21.15.64:0/487220443 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154034070 con 0x7fa16015e870 2024-03-07T07:19:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.507+0000 7fa15effd640 1 -- 172.21.15.64:0/487220443 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154030610 con 0x7fa16015e870 2024-03-07T07:19:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.507+0000 7fa15effd640 1 -- 172.21.15.64:0/487220443 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540309e0 con 0x7fa16015e870 2024-03-07T07:19:42.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.507+0000 7fa166497640 1 -- 172.21.15.64:0/487220443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 msgr2=0x7fa1600cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.507+0000 7fa166497640 1 --2- 172.21.15.64:0/487220443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600cf190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1540097a0 tx=0x7fa15402b960 comp rx=0 tx=0).stop 2024-03-07T07:19:42.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.508+0000 7fa166497640 1 -- 172.21.15.64:0/487220443 shutdown_connections 2024-03-07T07:19:42.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.508+0000 7fa166497640 1 --2- 172.21.15.64:0/487220443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600cf190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.508+0000 7fa166497640 1 -- 172.21.15.64:0/487220443 >> 172.21.15.64:0/487220443 conn(0x7fa1600d11e0 msgr2=0x7fa1600d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:42.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.508+0000 7fa166497640 1 -- 172.21.15.64:0/487220443 shutdown_connections 2024-03-07T07:19:42.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.508+0000 7fa166497640 1 -- 172.21.15.64:0/487220443 wait complete. 2024-03-07T07:19:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.509+0000 7fa166497640 1 Processor -- start 2024-03-07T07:19:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.509+0000 7fa166497640 1 -- start start 2024-03-07T07:19:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa166497640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1600cea70 0x7fa16014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa166497640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1600d3310 0x7fa1601451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa166497640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa166497640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600cf6d0 con 0x7fa16015e870 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa166497640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160070b60 con 0x7fa1600cea70 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa166497640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600cf850 con 0x7fa1600d3310 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59122/0 (socket says 172.21.15.64:59122) 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa15ffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1600cea70 0x7fa16014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 -- 172.21.15.64:0/2222635300 learned_addr learned my addr 172.21.15.64:0/2222635300 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa15f7fe640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1600d3310 0x7fa1601451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 -- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1600d3310 msgr2=0x7fa1601451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1600d3310 0x7fa1601451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 -- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1600cea70 msgr2=0x7fa16014f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1600cea70 0x7fa16014f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 -- 172.21.15.64:0/2222635300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160151f80 con 0x7fa16015e870 2024-03-07T07:19:42.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa164a34640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600c8ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c780 tx=0x7fa15800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa15cff9640 1 -- 172.21.15.64:0/2222635300 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158018070 con 0x7fa16015e870 2024-03-07T07:19:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa15cff9640 1 -- 172.21.15.64:0/2222635300 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15800a640 con 0x7fa16015e870 2024-03-07T07:19:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa160152260 con 0x7fa16015e870 2024-03-07T07:19:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.510+0000 7fa15cff9640 1 -- 172.21.15.64:0/2222635300 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15800aa70 con 0x7fa16015e870 2024-03-07T07:19:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.511+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1600cbec0 con 0x7fa16015e870 2024-03-07T07:19:42.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.511+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1600cc0c0 con 0x7fa16015e870 2024-03-07T07:19:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.512+0000 7fa15cff9640 1 -- 172.21.15.64:0/2222635300 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa15801b070 con 0x7fa16015e870 2024-03-07T07:19:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.512+0000 7fa15cff9640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa14c06b4d0 0x7fa14c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.512+0000 7fa15cff9640 1 -- 172.21.15.64:0/2222635300 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa158089760 con 0x7fa16015e870 2024-03-07T07:19:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.513+0000 7fa15ffff640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa14c06b4d0 0x7fa14c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.513+0000 7fa15ffff640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa14c06b4d0 0x7fa14c06d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa154036000 tx=0x7fa154008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.517+0000 7fa15cff9640 1 -- 172.21.15.64:0/2222635300 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa1600cc0c0 con 0x7fa16015e870 2024-03-07T07:19:42.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.696+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa160062c90 con 0x7fa16015e870 2024-03-07T07:19:42.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.696+0000 7fa15cff9640 1 -- 172.21.15.64:0/2222635300 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa158057f90 con 0x7fa16015e870 2024-03-07T07:19:42.696 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:42.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa14c06b4d0 msgr2=0x7fa14c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa14c06b4d0 0x7fa14c06d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa154036000 tx=0x7fa154008030 comp rx=0 tx=0).stop 2024-03-07T07:19:42.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 msgr2=0x7fa1600c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600c8ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c780 tx=0x7fa15800cc40 comp rx=0 tx=0).stop 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 shutdown_connections 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa14c06b4d0 0x7fa14c06d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa16015e870 0x7fa1600c8ea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1600d3310 0x7fa1601451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 --2- 172.21.15.64:0/2222635300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1600cea70 0x7fa16014f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 >> 172.21.15.64:0/2222635300 conn(0x7fa1600d11e0 msgr2=0x7fa1600b8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 shutdown_connections 2024-03-07T07:19:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.698+0000 7fa166497640 1 -- 172.21.15.64:0/2222635300 wait complete. 2024-03-07T07:19:42.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:42.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.813+0000 7f391c4dd640 1 Processor -- start 2024-03-07T07:19:42.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.813+0000 7f391c4dd640 1 -- start start 2024-03-07T07:19:42.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391c4dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39140ceef0 0x7f3914166d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391c4dd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140b59e0 con 0x7f391406cc30 2024-03-07T07:19:42.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391c4dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140b5cc0 con 0x7f3914145a20 2024-03-07T07:19:42.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391c4dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39140b6a80 con 0x7f39140ceef0 2024-03-07T07:19:42.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391aa7a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f39140ceef0 0x7f3914166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f3919a78640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f391406cc30 0x7f39140ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45968/0 (socket says 172.21.15.64:45968) 2024-03-07T07:19:42.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f3919a78640 1 -- 172.21.15.64:0/4207594188 learned_addr learned my addr 172.21.15.64:0/4207594188 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:42.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391aa7a640 1 -- 172.21.15.64:0/4207594188 >> v1:172.21.15.120:6789/0 conn(0x7f3914145a20 legacy=0x7f39140b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f3919277640 1 -- 172.21.15.64:0/4207594188 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240731283 0 0) 0x7f39140b5cc0 con 0x7f3914145a20 2024-03-07T07:19:42.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391aa7a640 1 -- 172.21.15.64:0/4207594188 >> v1:172.21.15.64:6789/0 conn(0x7f391406cc30 legacy=0x7f39140ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.814+0000 7f391aa7a640 1 -- 172.21.15.64:0/4207594188 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140b5fa0 con 0x7f39140ceef0 2024-03-07T07:19:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.815+0000 7f391aa7a640 1 --2- 172.21.15.64:0/4207594188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f3914166d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3904009d90 tx=0x7f390402bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b28f2edce5861d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.815+0000 7f3919277640 1 -- 172.21.15.64:0/4207594188 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904039070 con 0x7f39140ceef0 2024-03-07T07:19:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.815+0000 7f3919277640 1 -- 172.21.15.64:0/4207594188 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390402ca20 con 0x7f39140ceef0 2024-03-07T07:19:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.815+0000 7f3919277640 1 -- 172.21.15.64:0/4207594188 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390402cdf0 con 0x7f39140ceef0 2024-03-07T07:19:42.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.816+0000 7f391c4dd640 1 -- 172.21.15.64:0/4207594188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 msgr2=0x7f3914166d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.816+0000 7f391c4dd640 1 --2- 172.21.15.64:0/4207594188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f3914166d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3904009d90 tx=0x7f390402bcf0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.816+0000 7f391c4dd640 1 -- 172.21.15.64:0/4207594188 shutdown_connections 2024-03-07T07:19:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.816+0000 7f391c4dd640 1 --2- 172.21.15.64:0/4207594188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f3914166d90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.816+0000 7f391c4dd640 1 -- 172.21.15.64:0/4207594188 >> 172.21.15.64:0/4207594188 conn(0x7f391406b4d0 msgr2=0x7f391406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.816+0000 7f391c4dd640 1 -- 172.21.15.64:0/4207594188 shutdown_connections 2024-03-07T07:19:42.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.816+0000 7f391c4dd640 1 -- 172.21.15.64:0/4207594188 wait complete. 2024-03-07T07:19:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.817+0000 7f391c4dd640 1 Processor -- start 2024-03-07T07:19:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.817+0000 7f391c4dd640 1 -- start start 2024-03-07T07:19:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391c4dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391406cc30 0x7f391413f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391c4dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f391415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391c4dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3914145a20 0x7f391415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391c4dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140d1770 con 0x7f39140ceef0 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391c4dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140b6560 con 0x7f391406cc30 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391c4dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39140d18f0 con 0x7f3914145a20 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f391415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f391415e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59140/0 (socket says 172.21.15.64:59140) 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 -- 172.21.15.64:0/96307418 learned_addr learned my addr 172.21.15.64:0/96307418 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391a279640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391406cc30 0x7f391413f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f391aa7a640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3914145a20 0x7f391415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 -- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3914145a20 msgr2=0x7f391415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3914145a20 0x7f391415eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 -- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391406cc30 msgr2=0x7f391413f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391406cc30 0x7f391413f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:42.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 -- 172.21.15.64:0/96307418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391413fc70 con 0x7f39140ceef0 2024-03-07T07:19:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3919a78640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f391415e9b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f390800c780 tx=0x7f390800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.818+0000 7f3912ffd640 1 -- 172.21.15.64:0/96307418 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908018070 con 0x7f39140ceef0 2024-03-07T07:19:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.819+0000 7f3912ffd640 1 -- 172.21.15.64:0/96307418 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39080072d0 con 0x7f39140ceef0 2024-03-07T07:19:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.819+0000 7f3912ffd640 1 -- 172.21.15.64:0/96307418 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080047e0 con 0x7f39140ceef0 2024-03-07T07:19:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.819+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391414e290 con 0x7f39140ceef0 2024-03-07T07:19:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.819+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391414d7b0 con 0x7f39140ceef0 2024-03-07T07:19:42.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.820+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3914062c90 con 0x7f39140ceef0 2024-03-07T07:19:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.820+0000 7f3912ffd640 1 -- 172.21.15.64:0/96307418 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f390801b070 con 0x7f39140ceef0 2024-03-07T07:19:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.820+0000 7f3912ffd640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.820+0000 7f3912ffd640 1 -- 172.21.15.64:0/96307418 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3908089490 con 0x7f39140ceef0 2024-03-07T07:19:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.821+0000 7f391a279640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:42.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.821+0000 7f391a279640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f390006b4d0 0x7f390006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f390c000f60 tx=0x7f390c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:42.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:42.825+0000 7f3912ffd640 1 -- 172.21.15.64:0/96307418 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f391413fc70 con 0x7f39140ceef0 2024-03-07T07:19:43.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.004+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39140ce820 con 0x7f39140ceef0 2024-03-07T07:19:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.005+0000 7f3912ffd640 1 -- 172.21.15.64:0/96307418 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3908057e10 con 0x7f39140ceef0 2024-03-07T07:19:43.005 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f390006b4d0 msgr2=0x7f390006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f390006b4d0 0x7f390006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f390c000f60 tx=0x7f390c009040 comp rx=0 tx=0).stop 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 msgr2=0x7f391415e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f391415e9b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f390800c780 tx=0x7f390800cc40 comp rx=0 tx=0).stop 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 shutdown_connections 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39140ceef0 0x7f391415e9b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3914145a20 0x7f391415eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 --2- 172.21.15.64:0/96307418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f391406cc30 0x7f391413f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 >> 172.21.15.64:0/96307418 conn(0x7f391406b4d0 msgr2=0x7f39140744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 shutdown_connections 2024-03-07T07:19:43.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.007+0000 7f391c4dd640 1 -- 172.21.15.64:0/96307418 wait complete. 2024-03-07T07:19:43.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.122+0000 7f32aa799640 1 Processor -- start 2024-03-07T07:19:43.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.122+0000 7f32aa799640 1 -- start start 2024-03-07T07:19:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32aa799640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4070bc0 0x7f32a4071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32aa799640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32a40d26f0 0x7f32a40d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32aa799640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a4071610 con 0x7f32a4162110 2024-03-07T07:19:43.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32aa799640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a40718f0 con 0x7f32a40d26f0 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32aa799640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4071bd0 con 0x7f32a4070bc0 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a8d36640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32a40d26f0 0x7f32a40d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a8d36640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f32a40d26f0 0x7f32a40d2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59142/0 (socket says 172.21.15.64:59142) 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4070bc0 0x7f32a4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a8d36640 1 -- 172.21.15.64:0/2040092574 learned_addr learned my addr 172.21.15.64:0/2040092574 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a8d36640 1 -- 172.21.15.64:0/2040092574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4070bc0 msgr2=0x7f32a4071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a8d36640 1 --2- 172.21.15.64:0/2040092574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4070bc0 0x7f32a4071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a8d36640 1 -- 172.21.15.64:0/2040092574 >> v1:172.21.15.120:6789/0 conn(0x7f32a4162110 legacy=0x7f32a41624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a2ffd640 1 -- 172.21.15.64:0/2040092574 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464423524 0 0) 0x7f32a4071610 con 0x7f32a4162110 2024-03-07T07:19:43.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.123+0000 7f32a8d36640 1 -- 172.21.15.64:0/2040092574 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4071eb0 con 0x7f32a40d26f0 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.124+0000 7f32a8d36640 1 --2- 172.21.15.64:0/2040092574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a40d2b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f329c009e50 tx=0x7f329c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b719a0c70b80cf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.124+0000 7f32a2ffd640 1 -- 172.21.15.64:0/2040092574 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c02cb00 con 0x7f32a40d26f0 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.124+0000 7f32a2ffd640 1 -- 172.21.15.64:0/2040092574 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c02cca0 con 0x7f32a40d26f0 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.124+0000 7f32a2ffd640 1 -- 172.21.15.64:0/2040092574 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c034610 con 0x7f32a40d26f0 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 -- 172.21.15.64:0/2040092574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 msgr2=0x7f32a40d2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 --2- 172.21.15.64:0/2040092574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a40d2b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f329c009e50 tx=0x7f329c02bf60 comp rx=0 tx=0).stop 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 -- 172.21.15.64:0/2040092574 shutdown_connections 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 --2- 172.21.15.64:0/2040092574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a40d2b80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 --2- 172.21.15.64:0/2040092574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4070bc0 0x7f32a4071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 -- 172.21.15.64:0/2040092574 >> 172.21.15.64:0/2040092574 conn(0x7f32a40d0e10 msgr2=0x7f32a40d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 -- 172.21.15.64:0/2040092574 shutdown_connections 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.125+0000 7f32aa799640 1 -- 172.21.15.64:0/2040092574 wait complete. 2024-03-07T07:19:43.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.126+0000 7f32aa799640 1 Processor -- start 2024-03-07T07:19:43.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.126+0000 7f32aa799640 1 -- start start 2024-03-07T07:19:43.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32aa799640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32a4070bc0 0x7f32a40b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32aa799640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a41f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32aa799640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4162110 0x7f32a40b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32aa799640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4162f80 con 0x7f32a40d26f0 2024-03-07T07:19:43.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32aa799640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4162e00 con 0x7f32a4070bc0 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32aa799640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a415f8e0 con 0x7f32a4162110 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a41f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a41f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59144/0 (socket says 172.21.15.64:59144) 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 -- 172.21.15.64:0/2901003104 learned_addr learned my addr 172.21.15.64:0/2901003104 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32a4070bc0 0x7f32a40b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a8d36640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4162110 0x7f32a40b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 -- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4162110 msgr2=0x7f32a40b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4162110 0x7f32a40b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 -- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32a4070bc0 msgr2=0x7f32a40b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32a4070bc0 0x7f32a40b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 -- 172.21.15.64:0/2901003104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a40b3cd0 con 0x7f32a40d26f0 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a37fe640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a41f3d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f329400e8a0 tx=0x7f329400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a0ff9640 1 -- 172.21.15.64:0/2901003104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294017070 con 0x7f32a40d26f0 2024-03-07T07:19:43.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a0ff9640 1 -- 172.21.15.64:0/2901003104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400c940 con 0x7f32a40d26f0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a40b3fd0 con 0x7f32a40d26f0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.127+0000 7f32a0ff9640 1 -- 172.21.15.64:0/2901003104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400cd30 con 0x7f32a40d26f0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.128+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a40cb430 con 0x7f32a40d26f0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.128+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a4074810 con 0x7f32a40d26f0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.129+0000 7f32a0ff9640 1 -- 172.21.15.64:0/2901003104 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f329401a070 con 0x7f32a40d26f0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.129+0000 7f32a0ff9640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f329006b4d0 0x7f329006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.129+0000 7f32a0ff9640 1 -- 172.21.15.64:0/2901003104 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3294089fe0 con 0x7f32a40d26f0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.129+0000 7f32a3fff640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f329006b4d0 0x7f329006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.130+0000 7f32a3fff640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f329006b4d0 0x7f329006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3298000f60 tx=0x7f3298009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.134+0000 7f32a0ff9640 1 -- 172.21.15.64:0/2901003104 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f32a4074810 con 0x7f32a40d26f0 2024-03-07T07:19:43.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.314+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32a40cab80 con 0x7f32a40d26f0 2024-03-07T07:19:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.314+0000 7f32a0ff9640 1 -- 172.21.15.64:0/2901003104 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32a40b3fd0 con 0x7f32a40d26f0 2024-03-07T07:19:43.314 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f329006b4d0 msgr2=0x7f329006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f329006b4d0 0x7f329006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3298000f60 tx=0x7f3298009040 comp rx=0 tx=0).stop 2024-03-07T07:19:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 msgr2=0x7f32a41f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a41f3d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f329400e8a0 tx=0x7f329400ed60 comp rx=0 tx=0).stop 2024-03-07T07:19:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 shutdown_connections 2024-03-07T07:19:43.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f329006b4d0 0x7f329006d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a40d26f0 0x7f32a41f3d50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32a4162110 0x7f32a40b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 --2- 172.21.15.64:0/2901003104 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32a4070bc0 0x7f32a40b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 >> 172.21.15.64:0/2901003104 conn(0x7f32a40d0e10 msgr2=0x7f32a415d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 shutdown_connections 2024-03-07T07:19:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.316+0000 7f32aa799640 1 -- 172.21.15.64:0/2901003104 wait complete. 2024-03-07T07:19:43.327 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.429+0000 7f7203b1d640 1 Processor -- start 2024-03-07T07:19:43.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.429+0000 7f7203b1d640 1 -- start start 2024-03-07T07:19:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f7203b1d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f71fc15f170 0x7f71fc15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f7203b1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc072cf0 0x7f71fc071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f7203b1d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc071c50 con 0x7f71fc0d0990 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f7203b1d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc06c3a0 con 0x7f71fc15f170 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f7203b1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc071f30 con 0x7f71fc072cf0 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72010b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc072cf0 0x7f71fc071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72018b9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f71fc15f170 0x7f71fc15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72020ba640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f71fc0d0990 0x7f71fc1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:39666/0 (socket says 172.21.15.64:39666) 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72010b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc072cf0 0x7f71fc071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:43848/0 (socket says 172.21.15.64:43848) 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72020ba640 1 -- 172.21.15.64:0/3977095921 learned_addr learned my addr 172.21.15.64:0/3977095921 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72008b7640 1 -- 172.21.15.64:0/3977095921 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107437633 0 0) 0x7f71fc071c50 con 0x7f71fc0d0990 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72008b7640 1 -- 172.21.15.64:0/3977095921 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e8003520 con 0x7f71fc0d0990 2024-03-07T07:19:43.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72010b8640 1 -- 172.21.15.64:0/3977095921 >> v2:172.21.15.120:3300/0 conn(0x7f71fc15f170 msgr2=0x7f71fc15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72010b8640 1 --2- 172.21.15.64:0/3977095921 >> v2:172.21.15.120:3300/0 conn(0x7f71fc15f170 0x7f71fc15e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72010b8640 1 -- 172.21.15.64:0/3977095921 >> v1:172.21.15.120:6789/0 conn(0x7f71fc0d0990 legacy=0x7f71fc1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72010b8640 1 -- 172.21.15.64:0/3977095921 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc072210 con 0x7f71fc072cf0 2024-03-07T07:19:43.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.430+0000 7f72018b9640 1 --2- 172.21.15.64:0/3977095921 >> v2:172.21.15.120:3300/0 conn(0x7f71fc15f170 0x7f71fc15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.431+0000 7f72010b8640 1 --2- 172.21.15.64:0/3977095921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc072cf0 0x7f71fc071550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0097a0 tx=0x7f71ec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88904f67d25b420a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.431+0000 7f72008b7640 1 -- 172.21.15.64:0/3977095921 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec035020 con 0x7f71fc072cf0 2024-03-07T07:19:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.431+0000 7f72008b7640 1 -- 172.21.15.64:0/3977095921 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec031830 con 0x7f71fc072cf0 2024-03-07T07:19:43.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.431+0000 7f72008b7640 1 -- 172.21.15.64:0/3977095921 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec031c00 con 0x7f71fc072cf0 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 -- 172.21.15.64:0/3977095921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc072cf0 msgr2=0x7f71fc071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3977095921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc072cf0 0x7f71fc071550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0097a0 tx=0x7f71ec02b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 -- 172.21.15.64:0/3977095921 shutdown_connections 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3977095921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc072cf0 0x7f71fc071550 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3977095921 >> v2:172.21.15.120:3300/0 conn(0x7f71fc15f170 0x7f71fc15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 -- 172.21.15.64:0/3977095921 >> 172.21.15.64:0/3977095921 conn(0x7f71fc06b480 msgr2=0x7f71fc06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 -- 172.21.15.64:0/3977095921 shutdown_connections 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.432+0000 7f7203b1d640 1 -- 172.21.15.64:0/3977095921 wait complete. 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.433+0000 7f7203b1d640 1 Processor -- start 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.433+0000 7f7203b1d640 1 -- start start 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f7203b1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71fc072cf0 0x7f71fc158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f7203b1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc0d0990 0x7f71fc132730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f7203b1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71fc15f170 0x7f71fc1351f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f7203b1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0d0dd0 con 0x7f71fc15f170 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f7203b1d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0727d0 con 0x7f71fc072cf0 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f7203b1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0d0f50 con 0x7f71fc0d0990 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72020ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71fc15f170 0x7f71fc1351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72018b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71fc072cf0 0x7f71fc158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72010b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc0d0990 0x7f71fc132730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72018b9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71fc072cf0 0x7f71fc158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:41372/0 (socket says 172.21.15.64:41372) 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72018b9640 1 -- 172.21.15.64:0/3218047538 learned_addr learned my addr 172.21.15.64:0/3218047538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72010b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc0d0990 0x7f71fc132730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:43856/0 (socket says 172.21.15.64:43856) 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72020ba640 1 -- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc0d0990 msgr2=0x7f71fc132730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72020ba640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc0d0990 0x7f71fc132730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72020ba640 1 -- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71fc072cf0 msgr2=0x7f71fc158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72020ba640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71fc072cf0 0x7f71fc158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72020ba640 1 -- 172.21.15.64:0/3218047538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc132e60 con 0x7f71fc15f170 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72010b8640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc0d0990 0x7f71fc132730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:43.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72018b9640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71fc072cf0 0x7f71fc158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f72020ba640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71fc15f170 0x7f71fc1351f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ca80 tx=0x7f71f400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f71f27fc640 1 -- 172.21.15.64:0/3218047538 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4017070 con 0x7f71fc15f170 2024-03-07T07:19:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f71f27fc640 1 -- 172.21.15.64:0/3218047538 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f400ac30 con 0x7f71fc15f170 2024-03-07T07:19:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc133140 con 0x7f71fc15f170 2024-03-07T07:19:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.434+0000 7f71f27fc640 1 -- 172.21.15.64:0/3218047538 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4012600 con 0x7f71fc15f170 2024-03-07T07:19:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.435+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc133700 con 0x7f71fc15f170 2024-03-07T07:19:43.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.435+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc161660 con 0x7f71fc15f170 2024-03-07T07:19:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.436+0000 7f71f27fc640 1 -- 172.21.15.64:0/3218047538 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f71f401a070 con 0x7f71fc15f170 2024-03-07T07:19:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.436+0000 7f71f27fc640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f71e006b4d0 0x7f71e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.436+0000 7f71f27fc640 1 -- 172.21.15.64:0/3218047538 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71f4089eb0 con 0x7f71fc15f170 2024-03-07T07:19:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.436+0000 7f72018b9640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f71e006b4d0 0x7f71e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.437+0000 7f72018b9640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f71e006b4d0 0x7f71e006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71f80095e0 tx=0x7f71f8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.441+0000 7f71f27fc640 1 -- 172.21.15.64:0/3218047538 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f71fc161660 con 0x7f71fc15f170 2024-03-07T07:19:43.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.621+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71fc062c90 con 0x7f71fc15f170 2024-03-07T07:19:43.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.623+0000 7f71f27fc640 1 -- 172.21.15.64:0/3218047538 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f71f400ae20 con 0x7f71fc15f170 2024-03-07T07:19:43.623 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:43.623 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.624+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f71e006b4d0 msgr2=0x7f71e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.624+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f71e006b4d0 0x7f71e006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71f80095e0 tx=0x7f71f8009190 comp rx=0 tx=0).stop 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.624+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71fc15f170 msgr2=0x7f71fc1351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.624+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71fc15f170 0x7f71fc1351f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ca80 tx=0x7f71f400cf40 comp rx=0 tx=0).stop 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 shutdown_connections 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f71e006b4d0 0x7f71e006d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71fc15f170 0x7f71fc1351f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f71fc0d0990 0x7f71fc132730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 --2- 172.21.15.64:0/3218047538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71fc072cf0 0x7f71fc158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 >> 172.21.15.64:0/3218047538 conn(0x7f71fc06b480 msgr2=0x7f71fc0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:43.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 shutdown_connections 2024-03-07T07:19:43.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.625+0000 7f7203b1d640 1 -- 172.21.15.64:0/3218047538 wait complete. 2024-03-07T07:19:43.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:43.636 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:43.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.740+0000 7f996ae34640 1 Processor -- start 2024-03-07T07:19:43.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.740+0000 7f996ae34640 1 -- start start 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f996ae34640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9964162110 0x7f99641624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f996ae34640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9964070bc0 0x7f996415e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f996ae34640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9964162a30 con 0x7f9964071050 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f996ae34640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9964071610 con 0x7f9964162110 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f996ae34640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99640718f0 con 0x7f9964070bc0 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f9968bd0640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9964162110 0x7f99641624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f99693d1640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9964071050 0x7f996415ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45970/0 (socket says 172.21.15.64:45970) 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f99693d1640 1 -- 172.21.15.64:0/3915131976 learned_addr learned my addr 172.21.15.64:0/3915131976 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f9963fff640 1 --2- 172.21.15.64:0/3915131976 >> v2:172.21.15.120:3300/0 conn(0x7f9964070bc0 0x7f996415e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f9968bd0640 1 -- 172.21.15.64:0/3915131976 >> v2:172.21.15.120:3300/0 conn(0x7f9964070bc0 msgr2=0x7f996415e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:43.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f9968bd0640 1 --2- 172.21.15.64:0/3915131976 >> v2:172.21.15.120:3300/0 conn(0x7f9964070bc0 0x7f996415e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f9968bd0640 1 -- 172.21.15.64:0/3915131976 >> v1:172.21.15.64:6789/0 conn(0x7f9964071050 legacy=0x7f996415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f99637fe640 1 -- 172.21.15.64:0/3915131976 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024592076 0 0) 0x7f9964162a30 con 0x7f9964071050 2024-03-07T07:19:43.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.741+0000 7f9968bd0640 1 -- 172.21.15.64:0/3915131976 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964071bd0 con 0x7f9964162110 2024-03-07T07:19:43.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.742+0000 7f9968bd0640 1 --2- 172.21.15.64:0/3915131976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99641624f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f996408d010 tx=0x7f995802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c033648bdc0716 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.742+0000 7f99637fe640 1 -- 172.21.15.64:0/3915131976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99580315a0 con 0x7f9964162110 2024-03-07T07:19:43.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.742+0000 7f99637fe640 1 -- 172.21.15.64:0/3915131976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9958037070 con 0x7f9964162110 2024-03-07T07:19:43.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.742+0000 7f99637fe640 1 -- 172.21.15.64:0/3915131976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958031960 con 0x7f9964162110 2024-03-07T07:19:43.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 -- 172.21.15.64:0/3915131976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 msgr2=0x7f99641624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 --2- 172.21.15.64:0/3915131976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99641624f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f996408d010 tx=0x7f995802b840 comp rx=0 tx=0).stop 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 -- 172.21.15.64:0/3915131976 shutdown_connections 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 --2- 172.21.15.64:0/3915131976 >> v2:172.21.15.120:3300/0 conn(0x7f9964070bc0 0x7f996415e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 --2- 172.21.15.64:0/3915131976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99641624f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 -- 172.21.15.64:0/3915131976 >> 172.21.15.64:0/3915131976 conn(0x7f99640d0e10 msgr2=0x7f99640d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 -- 172.21.15.64:0/3915131976 shutdown_connections 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.743+0000 7f996ae34640 1 -- 172.21.15.64:0/3915131976 wait complete. 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.744+0000 7f996ae34640 1 Processor -- start 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.744+0000 7f996ae34640 1 -- start start 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f996ae34640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9964070bc0 0x7f9964155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f996ae34640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9964071050 0x7f99641562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f996ae34640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99640b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f996ae34640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964162e00 con 0x7f9964162110 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f996ae34640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99640d21d0 con 0x7f9964071050 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f996ae34640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9964162f80 con 0x7f9964070bc0 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99640b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99640b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59166/0 (socket says 172.21.15.64:59166) 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 -- 172.21.15.64:0/1921441958 learned_addr learned my addr 172.21.15.64:0/1921441958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f9963fff640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9964071050 0x7f99641562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f9968bd0640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9964070bc0 0x7f9964155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 -- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9964070bc0 msgr2=0x7f9964155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9964070bc0 0x7f9964155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 -- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9964071050 msgr2=0x7f99641562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9964071050 0x7f99641562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 -- 172.21.15.64:0/1921441958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99640cb1c0 con 0x7f9964162110 2024-03-07T07:19:43.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99693d1640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99640b3240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c7d0 tx=0x7f995c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99617fa640 1 -- 172.21.15.64:0/1921441958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c018070 con 0x7f9964162110 2024-03-07T07:19:43.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99617fa640 1 -- 172.21.15.64:0/1921441958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c00a640 con 0x7f9964162110 2024-03-07T07:19:43.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f99617fa640 1 -- 172.21.15.64:0/1921441958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00aa70 con 0x7f9964162110 2024-03-07T07:19:43.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.745+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99640cb4a0 con 0x7f9964162110 2024-03-07T07:19:43.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.746+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9964156040 con 0x7f9964162110 2024-03-07T07:19:43.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.746+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964062c90 con 0x7f9964162110 2024-03-07T07:19:43.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.747+0000 7f99617fa640 1 -- 172.21.15.64:0/1921441958 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f995c01b070 con 0x7f9964162110 2024-03-07T07:19:43.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.747+0000 7f99617fa640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:43.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.747+0000 7f99617fa640 1 -- 172.21.15.64:0/1921441958 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f995c0896e0 con 0x7f9964162110 2024-03-07T07:19:43.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.747+0000 7f9968bd0640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:43.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.748+0000 7f9968bd0640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f994806b4d0 0x7f994806d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9958036000 tx=0x7f995803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:43.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.752+0000 7f99617fa640 1 -- 172.21.15.64:0/1921441958 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f995c057fa0 con 0x7f9964162110 2024-03-07T07:19:43.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.931+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99640706d0 con 0x7f9964162110 2024-03-07T07:19:43.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.932+0000 7f99617fa640 1 -- 172.21.15.64:0/1921441958 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f995c05c040 con 0x7f9964162110 2024-03-07T07:19:43.932 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f994806b4d0 msgr2=0x7f994806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f994806b4d0 0x7f994806d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9958036000 tx=0x7f995803d040 comp rx=0 tx=0).stop 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 msgr2=0x7f99640b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99640b3240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c7d0 tx=0x7f995c00cc90 comp rx=0 tx=0).stop 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 shutdown_connections 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9964162110 0x7f99640b3240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9964071050 0x7f99641562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 --2- 172.21.15.64:0/1921441958 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9964070bc0 0x7f9964155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 >> 172.21.15.64:0/1921441958 conn(0x7f99640d0e10 msgr2=0x7f996415fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 shutdown_connections 2024-03-07T07:19:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:43.934+0000 7f996ae34640 1 -- 172.21.15.64:0/1921441958 wait complete. 2024-03-07T07:19:43.945 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:44.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.049+0000 7fb4c20a1640 1 Processor -- start 2024-03-07T07:19:44.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.049+0000 7fb4c20a1640 1 -- start start 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4c20a1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb4bc158780 0x7fb4bc15f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4c20a1640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc070530 con 0x7fb4bc06ea50 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4c20a1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc070810 con 0x7fb4bc0af950 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4c20a1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc0b0110 con 0x7fb4bc158780 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4bb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb4bc158780 0x7fb4bc15f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4bbfff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb4bc06ea50 0x7fb4bc0b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45976/0 (socket says 172.21.15.64:45976) 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4bbfff640 1 -- 172.21.15.64:0/2321315401 learned_addr learned my addr 172.21.15.64:0/2321315401 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4bb7fe640 1 -- 172.21.15.64:0/2321315401 >> v1:172.21.15.120:6789/0 conn(0x7fb4bc0af950 legacy=0x7fb4bc0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4bb7fe640 1 -- 172.21.15.64:0/2321315401 >> v1:172.21.15.64:6789/0 conn(0x7fb4bc06ea50 legacy=0x7fb4bc0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4ba7fc640 1 -- 172.21.15.64:0/2321315401 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446353622 0 0) 0x7fb4bc070810 con 0x7fb4bc0af950 2024-03-07T07:19:44.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.050+0000 7fb4bb7fe640 1 -- 172.21.15.64:0/2321315401 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc070af0 con 0x7fb4bc158780 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.051+0000 7fb4bb7fe640 1 --2- 172.21.15.64:0/2321315401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc158780 0x7fb4bc15f2c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0b0290 tx=0x7fb4ac02b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2e8ee6ae98f517 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.051+0000 7fb4ba7fc640 1 -- 172.21.15.64:0/2321315401 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac037070 con 0x7fb4bc158780 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.051+0000 7fb4ba7fc640 1 -- 172.21.15.64:0/2321315401 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac030710 con 0x7fb4bc158780 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.051+0000 7fb4ba7fc640 1 -- 172.21.15.64:0/2321315401 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac030ae0 con 0x7fb4bc158780 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.051+0000 7fb4c20a1640 1 -- 172.21.15.64:0/2321315401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc158780 msgr2=0x7fb4bc15f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.051+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/2321315401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc158780 0x7fb4bc15f2c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0b0290 tx=0x7fb4ac02b6f0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.052+0000 7fb4c20a1640 1 -- 172.21.15.64:0/2321315401 shutdown_connections 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.052+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/2321315401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc158780 0x7fb4bc15f2c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.052+0000 7fb4c20a1640 1 -- 172.21.15.64:0/2321315401 >> 172.21.15.64:0/2321315401 conn(0x7fb4bc06c220 msgr2=0x7fb4bc0b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:44.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.052+0000 7fb4c20a1640 1 -- 172.21.15.64:0/2321315401 shutdown_connections 2024-03-07T07:19:44.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.052+0000 7fb4c20a1640 1 -- 172.21.15.64:0/2321315401 wait complete. 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.053+0000 7fb4c20a1640 1 Processor -- start 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.053+0000 7fb4c20a1640 1 -- start start 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc06ea50 0x7fb4bc153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4bc0af950 0x7fb4bc159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4bc158780 0x7fb4bc0cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc071a00 con 0x7fb4bc06ea50 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc070e60 con 0x7fb4bc0af950 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc158b60 con 0x7fb4bc158780 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc06ea50 0x7fb4bc153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc06ea50 0x7fb4bc153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59190/0 (socket says 172.21.15.64:59190) 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 -- 172.21.15.64:0/380618315 learned_addr learned my addr 172.21.15.64:0/380618315 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:44.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4baffd640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4bc0af950 0x7fb4bc159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bbfff640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4bc158780 0x7fb4bc0cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 -- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4bc158780 msgr2=0x7fb4bc0cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4bc158780 0x7fb4bc0cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 -- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4bc0af950 msgr2=0x7fb4bc159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4bc0af950 0x7fb4bc159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 -- 172.21.15.64:0/380618315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc15a4e0 con 0x7fb4bc06ea50 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bbfff640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4bc158780 0x7fb4bc0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4baffd640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4bc0af950 0x7fb4bc159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4bb7fe640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc06ea50 0x7fb4bc153c60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009890 tx=0x7fb4ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4a3fff640 1 -- 172.21.15.64:0/380618315 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac042070 con 0x7fb4bc06ea50 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4a3fff640 1 -- 172.21.15.64:0/380618315 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac032a40 con 0x7fb4bc06ea50 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4a3fff640 1 -- 172.21.15.64:0/380618315 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac037040 con 0x7fb4bc06ea50 2024-03-07T07:19:44.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc0b0580 con 0x7fb4bc06ea50 2024-03-07T07:19:44.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.054+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc1541a0 con 0x7fb4bc06ea50 2024-03-07T07:19:44.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.055+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc0b0370 con 0x7fb4bc06ea50 2024-03-07T07:19:44.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.056+0000 7fb4a3fff640 1 -- 172.21.15.64:0/380618315 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb4ac045070 con 0x7fb4bc06ea50 2024-03-07T07:19:44.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.056+0000 7fb4a3fff640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb4a806b500 0x7fb4a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.056+0000 7fb4a3fff640 1 -- 172.21.15.64:0/380618315 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4ac0ac2c0 con 0x7fb4bc06ea50 2024-03-07T07:19:44.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.056+0000 7fb4baffd640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb4a806b500 0x7fb4a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.057+0000 7fb4baffd640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb4a806b500 0x7fb4a806d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc1f7230 tx=0x7fb4b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.060+0000 7fb4a3fff640 1 -- 172.21.15.64:0/380618315 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb4ac07b2f0 con 0x7fb4bc06ea50 2024-03-07T07:19:44.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.241+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4bc0ccbe0 con 0x7fb4bc06ea50 2024-03-07T07:19:44.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.242+0000 7fb4a3fff640 1 -- 172.21.15.64:0/380618315 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4ac07abf0 con 0x7fb4bc06ea50 2024-03-07T07:19:44.242 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb4a806b500 msgr2=0x7fb4a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb4a806b500 0x7fb4a806d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc1f7230 tx=0x7fb4b0009040 comp rx=0 tx=0).stop 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc06ea50 msgr2=0x7fb4bc153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc06ea50 0x7fb4bc153c60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009890 tx=0x7fb4ac0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 shutdown_connections 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb4a806b500 0x7fb4a806d9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4bc158780 0x7fb4bc0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4bc0af950 0x7fb4bc159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 --2- 172.21.15.64:0/380618315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4bc06ea50 0x7fb4bc153c60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 >> 172.21.15.64:0/380618315 conn(0x7fb4bc06c220 msgr2=0x7fb4bc160650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:44.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 shutdown_connections 2024-03-07T07:19:44.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.244+0000 7fb4c20a1640 1 -- 172.21.15.64:0/380618315 wait complete. 2024-03-07T07:19:44.254 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:44.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.374+0000 7f091e4a5640 1 Processor -- start 2024-03-07T07:19:44.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.374+0000 7f091e4a5640 1 -- start start 2024-03-07T07:19:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.375+0000 7f091e4a5640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0918161990 0x7f09180b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.375+0000 7f091e4a5640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091806d280 con 0x7f09180b1040 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.375+0000 7f091e4a5640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091806c400 con 0x7f091809b0e0 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.375+0000 7f091e4a5640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09180b1770 con 0x7f0918161990 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.375+0000 7f0917fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0918161990 0x7f09180b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.375+0000 7f09177fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f09180b1040 0x7f091809a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45982/0 (socket says 172.21.15.64:45982) 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.375+0000 7f09177fe640 1 -- 172.21.15.64:0/625158492 learned_addr learned my addr 172.21.15.64:0/625158492 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.376+0000 7f0917fff640 1 -- 172.21.15.64:0/625158492 >> v1:172.21.15.120:6789/0 conn(0x7f091809b0e0 legacy=0x7f09180cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.376+0000 7f0917fff640 1 -- 172.21.15.64:0/625158492 >> v1:172.21.15.64:6789/0 conn(0x7f09180b1040 legacy=0x7f091809a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.376+0000 7f0917fff640 1 -- 172.21.15.64:0/625158492 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091806c6e0 con 0x7f0918161990 2024-03-07T07:19:44.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.376+0000 7f0917fff640 1 --2- 172.21.15.64:0/625158492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f09180b46b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09080097a0 tx=0x7f090802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09019baa6d597f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.377+0000 7f0916ffd640 1 -- 172.21.15.64:0/625158492 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908035020 con 0x7f0918161990 2024-03-07T07:19:44.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.377+0000 7f0916ffd640 1 -- 172.21.15.64:0/625158492 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0908031830 con 0x7f0918161990 2024-03-07T07:19:44.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.377+0000 7f0916ffd640 1 -- 172.21.15.64:0/625158492 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908031c00 con 0x7f0918161990 2024-03-07T07:19:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.378+0000 7f091e4a5640 1 -- 172.21.15.64:0/625158492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 msgr2=0x7f09180b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.378+0000 7f091e4a5640 1 --2- 172.21.15.64:0/625158492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f09180b46b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09080097a0 tx=0x7f090802b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.378+0000 7f091e4a5640 1 -- 172.21.15.64:0/625158492 shutdown_connections 2024-03-07T07:19:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.378+0000 7f091e4a5640 1 --2- 172.21.15.64:0/625158492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f09180b46b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.378+0000 7f091e4a5640 1 -- 172.21.15.64:0/625158492 >> 172.21.15.64:0/625158492 conn(0x7f09180b3b90 msgr2=0x7f09180b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.378+0000 7f091e4a5640 1 -- 172.21.15.64:0/625158492 shutdown_connections 2024-03-07T07:19:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.378+0000 7f091e4a5640 1 -- 172.21.15.64:0/625158492 wait complete. 2024-03-07T07:19:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.379+0000 7f091e4a5640 1 Processor -- start 2024-03-07T07:19:44.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.379+0000 7f091e4a5640 1 -- start start 2024-03-07T07:19:44.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091e4a5640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f091809b0e0 0x7f091815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091e4a5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09180b1040 0x7f09180b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091e4a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f091809e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091e4a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09180d0770 con 0x7f0918161990 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091e4a5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09180d05f0 con 0x7f09180b1040 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091e4a5640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09180d08f0 con 0x7f091809b0e0 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091ca42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f091809e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f0917fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f091809b0e0 0x7f091815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091ca42640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f091809e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59214/0 (socket says 172.21.15.64:59214) 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f091ca42640 1 -- 172.21.15.64:0/32246148 learned_addr learned my addr 172.21.15.64:0/32246148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.380+0000 7f09177fe640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09180b1040 0x7f09180b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091ca42640 1 -- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f091809b0e0 msgr2=0x7f091815d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091ca42640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f091809b0e0 0x7f091815d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091ca42640 1 -- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09180b1040 msgr2=0x7f09180b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091ca42640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09180b1040 0x7f09180b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091ca42640 1 -- 172.21.15.64:0/32246148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091809cf00 con 0x7f0918161990 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f0917fff640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f091809b0e0 0x7f091815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f09177fe640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09180b1040 0x7f09180b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091ca42640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f091809e910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f091000c760 tx=0x7f091000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f0914ff9640 1 -- 172.21.15.64:0/32246148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0910003b70 con 0x7f0918161990 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f0914ff9640 1 -- 172.21.15.64:0/32246148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0910003d10 con 0x7f0918161990 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091809d1e0 con 0x7f0918161990 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f0914ff9640 1 -- 172.21.15.64:0/32246148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091000aa90 con 0x7f0918161990 2024-03-07T07:19:44.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.381+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091809d7a0 con 0x7f0918161990 2024-03-07T07:19:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.382+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0918062c90 con 0x7f0918161990 2024-03-07T07:19:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.383+0000 7f0914ff9640 1 -- 172.21.15.64:0/32246148 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f091000ac30 con 0x7f0918161990 2024-03-07T07:19:44.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.383+0000 7f0914ff9640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f090406b4d0 0x7f090406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.383+0000 7f0914ff9640 1 -- 172.21.15.64:0/32246148 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0910089ef0 con 0x7f0918161990 2024-03-07T07:19:44.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.383+0000 7f0917fff640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f090406b4d0 0x7f090406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.384+0000 7f0917fff640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f090406b4d0 0x7f090406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0908036000 tx=0x7f090803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.388+0000 7f0914ff9640 1 -- 172.21.15.64:0/32246148 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f09100587b0 con 0x7f0918161990 2024-03-07T07:19:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.568+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0918072170 con 0x7f0918161990 2024-03-07T07:19:44.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.569+0000 7f0914ff9640 1 -- 172.21.15.64:0/32246148 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f091005c850 con 0x7f0918161990 2024-03-07T07:19:44.569 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f090406b4d0 msgr2=0x7f090406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f090406b4d0 0x7f090406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0908036000 tx=0x7f090803c040 comp rx=0 tx=0).stop 2024-03-07T07:19:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 msgr2=0x7f091809e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f091809e910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f091000c760 tx=0x7f091000cc20 comp rx=0 tx=0).stop 2024-03-07T07:19:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 shutdown_connections 2024-03-07T07:19:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f090406b4d0 0x7f090406d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0918161990 0x7f091809e910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09180b1040 0x7f09180b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 --2- 172.21.15.64:0/32246148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f091809b0e0 0x7f091815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 >> 172.21.15.64:0/32246148 conn(0x7f09180b3b90 msgr2=0x7f0918160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:44.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 shutdown_connections 2024-03-07T07:19:44.572 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.571+0000 7f091e4a5640 1 -- 172.21.15.64:0/32246148 wait complete. 2024-03-07T07:19:44.606 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-03-07T07:19:44.606 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.708+0000 7fd1d8887640 1 Processor -- start 2024-03-07T07:19:44.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.708+0000 7fd1d8887640 1 -- start start 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d8887640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1d40cfad0 0x7fd1d4166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d8887640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d40d0bc0 con 0x7fd1d40b4f10 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d8887640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d40d0ea0 con 0x7fd1d40b7360 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d8887640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d40d0970 con 0x7fd1d40cfad0 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d2d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1d40cfad0 0x7fd1d4166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d259c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd1d40b4f10 0x7fd1d415cef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45998/0 (socket says 172.21.15.64:45998) 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d259c640 1 -- 172.21.15.64:0/3796718688 learned_addr learned my addr 172.21.15.64:0/3796718688 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d1d9b640 1 -- 172.21.15.64:0/3796718688 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010592142 0 0) 0x7fd1d40d0ea0 con 0x7fd1d40b7360 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d2d9d640 1 -- 172.21.15.64:0/3796718688 >> v1:172.21.15.120:6789/0 conn(0x7fd1d40b7360 legacy=0x7fd1d40b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d2d9d640 1 -- 172.21.15.64:0/3796718688 >> v1:172.21.15.64:6789/0 conn(0x7fd1d40b4f10 legacy=0x7fd1d415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.709+0000 7fd1d2d9d640 1 -- 172.21.15.64:0/3796718688 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d40d1180 con 0x7fd1d40cfad0 2024-03-07T07:19:44.710 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d2d9d640 1 --2- 172.21.15.64:0/3796718688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4166ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009d90 tx=0x7fd1cc02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2b927fbc92f738 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d1d9b640 1 -- 172.21.15.64:0/3796718688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc039070 con 0x7fd1d40cfad0 2024-03-07T07:19:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d1d9b640 1 -- 172.21.15.64:0/3796718688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc02cab0 con 0x7fd1d40cfad0 2024-03-07T07:19:44.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d1d9b640 1 -- 172.21.15.64:0/3796718688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc034440 con 0x7fd1d40cfad0 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d8887640 1 -- 172.21.15.64:0/3796718688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 msgr2=0x7fd1d4166ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3796718688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4166ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009d90 tx=0x7fd1cc02bd80 comp rx=0 tx=0).stop 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d8887640 1 -- 172.21.15.64:0/3796718688 shutdown_connections 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.710+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3796718688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4166ae0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.711+0000 7fd1d8887640 1 -- 172.21.15.64:0/3796718688 >> 172.21.15.64:0/3796718688 conn(0x7fd1d406b480 msgr2=0x7fd1d406b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.711+0000 7fd1d8887640 1 -- 172.21.15.64:0/3796718688 shutdown_connections 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.711+0000 7fd1d8887640 1 -- 172.21.15.64:0/3796718688 wait complete. 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 Processor -- start 2024-03-07T07:19:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 -- start start 2024-03-07T07:19:44.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1d40b4f10 0x7fd1d414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d40b7360 0x7fd1d415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d4159d10 con 0x7fd1d40cfad0 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d40d1740 con 0x7fd1d40b7360 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.712+0000 7fd1d8887640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d4159e90 con 0x7fd1d40b4f10 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59228/0 (socket says 172.21.15.64:59228) 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 -- 172.21.15.64:0/3141510820 learned_addr learned my addr 172.21.15.64:0/3141510820 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d259c640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1d40b4f10 0x7fd1d414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1cbfff640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d40b7360 0x7fd1d415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 -- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1d40b4f10 msgr2=0x7fd1d414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1d40b4f10 0x7fd1d414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 -- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d40b7360 msgr2=0x7fd1d415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d40b7360 0x7fd1d415e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 -- 172.21.15.64:0/3141510820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d414f810 con 0x7fd1d40cfad0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d2d9d640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4153ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009d90 tx=0x7fd1cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1caffd640 1 -- 172.21.15.64:0/3141510820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc044030 con 0x7fd1d40cfad0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1caffd640 1 -- 172.21.15.64:0/3141510820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc02c8b0 con 0x7fd1d40cfad0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d414faf0 con 0x7fd1d40cfad0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1caffd640 1 -- 172.21.15.64:0/3141510820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc039040 con 0x7fd1d40cfad0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.713+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d41500b0 con 0x7fd1d40cfad0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.714+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d40b6230 con 0x7fd1d40cfad0 2024-03-07T07:19:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.715+0000 7fd1caffd640 1 -- 172.21.15.64:0/3141510820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd1cc047070 con 0x7fd1d40cfad0 2024-03-07T07:19:44.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.715+0000 7fd1caffd640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd1b406b4d0 0x7fd1b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:44.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.715+0000 7fd1caffd640 1 -- 172.21.15.64:0/3141510820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1cc0ac930 con 0x7fd1d40cfad0 2024-03-07T07:19:44.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.715+0000 7fd1d259c640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd1b406b4d0 0x7fd1b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:44.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.716+0000 7fd1d259c640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd1b406b4d0 0x7fd1b406d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000f60 tx=0x7fd1c4007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:44.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.719+0000 7fd1caffd640 1 -- 172.21.15.64:0/3141510820 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd1d40b6230 con 0x7fd1d40cfad0 2024-03-07T07:19:44.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.901+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1d4062c90 con 0x7fd1d40cfad0 2024-03-07T07:19:44.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.902+0000 7fd1caffd640 1 -- 172.21.15.64:0/3141510820 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fd1cc04e070 con 0x7fd1d40cfad0 2024-03-07T07:19:44.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:44.902 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.903+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd1b406b4d0 msgr2=0x7fd1b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.903+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd1b406b4d0 0x7fd1b406d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000f60 tx=0x7fd1c4007e00 comp rx=0 tx=0).stop 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.903+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 msgr2=0x7fd1d4153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.903+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4153ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009d90 tx=0x7fd1cc004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 shutdown_connections 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd1b406b4d0 0x7fd1b406d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d40cfad0 0x7fd1d4153ce0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1d40b7360 0x7fd1d415e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 --2- 172.21.15.64:0/3141510820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1d40b4f10 0x7fd1d414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:44.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 >> 172.21.15.64:0/3141510820 conn(0x7fd1d406b480 msgr2=0x7fd1d40740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:44.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 shutdown_connections 2024-03-07T07:19:44.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:44.904+0000 7fd1d8887640 1 -- 172.21.15.64:0/3141510820 wait complete. 2024-03-07T07:19:44.914 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:44.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:45.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.018+0000 7fa3a37b8640 1 Processor -- start 2024-03-07T07:19:45.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.019+0000 7fa3a37b8640 1 -- start start 2024-03-07T07:19:45.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.019+0000 7fa3a37b8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa39c141360 0x7fa39c141910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.019+0000 7fa3a37b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b44f0 0x7fa39c1620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.019+0000 7fa3a37b8640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c15ebd0 con 0x7fa39c0b5720 2024-03-07T07:19:45.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.019+0000 7fa3a37b8640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c15eeb0 con 0x7fa39c141360 2024-03-07T07:19:45.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.019+0000 7fa3a37b8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c15f190 con 0x7fa39c0b44f0 2024-03-07T07:19:45.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a1554640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa39c141360 0x7fa39c141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a0d53640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa39c0b5720 0x7fa39c0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46012/0 (socket says 172.21.15.64:46012) 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a0d53640 1 -- 172.21.15.64:0/4254581365 learned_addr learned my addr 172.21.15.64:0/4254581365 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a1d55640 1 --2- 172.21.15.64:0/4254581365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b44f0 0x7fa39c1620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa39bfff640 1 -- 172.21.15.64:0/4254581365 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135666775 0 0) 0x7fa39c15ebd0 con 0x7fa39c0b5720 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa39bfff640 1 -- 172.21.15.64:0/4254581365 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa384003520 con 0x7fa39c0b5720 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a1d55640 1 -- 172.21.15.64:0/4254581365 >> v2:172.21.15.64:3300/0 conn(0x7fa39c141360 msgr2=0x7fa39c141910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a1d55640 1 --2- 172.21.15.64:0/4254581365 >> v2:172.21.15.64:3300/0 conn(0x7fa39c141360 0x7fa39c141910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a1d55640 1 -- 172.21.15.64:0/4254581365 >> v1:172.21.15.64:6789/0 conn(0x7fa39c0b5720 legacy=0x7fa39c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a1d55640 1 -- 172.21.15.64:0/4254581365 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c15f470 con 0x7fa39c0b44f0 2024-03-07T07:19:45.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.020+0000 7fa3a1554640 1 --2- 172.21.15.64:0/4254581365 >> v2:172.21.15.64:3300/0 conn(0x7fa39c141360 0x7fa39c141910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa3a1d55640 1 --2- 172.21.15.64:0/4254581365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b44f0 0x7fa39c1620f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0097a0 tx=0x7fa38c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d0397d7f9be2e52 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa39bfff640 1 -- 172.21.15.64:0/4254581365 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c035020 con 0x7fa39c0b44f0 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa39bfff640 1 -- 172.21.15.64:0/4254581365 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c031810 con 0x7fa39c0b44f0 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa39bfff640 1 -- 172.21.15.64:0/4254581365 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c031be0 con 0x7fa39c0b44f0 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa3a37b8640 1 -- 172.21.15.64:0/4254581365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b44f0 msgr2=0x7fa39c1620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/4254581365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b44f0 0x7fa39c1620f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0097a0 tx=0x7fa38c02b8b0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa3a37b8640 1 -- 172.21.15.64:0/4254581365 shutdown_connections 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/4254581365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b44f0 0x7fa39c1620f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/4254581365 >> v2:172.21.15.64:3300/0 conn(0x7fa39c141360 0x7fa39c141910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.021+0000 7fa3a37b8640 1 -- 172.21.15.64:0/4254581365 >> 172.21.15.64:0/4254581365 conn(0x7fa39c0705e0 msgr2=0x7fa39c0709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.022+0000 7fa3a37b8640 1 -- 172.21.15.64:0/4254581365 shutdown_connections 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.022+0000 7fa3a37b8640 1 -- 172.21.15.64:0/4254581365 wait complete. 2024-03-07T07:19:45.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 Processor -- start 2024-03-07T07:19:45.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 -- start start 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa39c0b44f0 0x7fa39c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b5720 0x7fa39c13ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39c141360 0x7fa39c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0b5bd0 con 0x7fa39c141360 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c15e8c0 con 0x7fa39c0b44f0 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.023+0000 7fa3a37b8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0b5d50 con 0x7fa39c0b5720 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39c141360 0x7fa39c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39c141360 0x7fa39c15d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59248/0 (socket says 172.21.15.64:59248) 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 -- 172.21.15.64:0/34626261 learned_addr learned my addr 172.21.15.64:0/34626261 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1554640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa39c0b44f0 0x7fa39c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a0d53640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b5720 0x7fa39c13ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 -- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b5720 msgr2=0x7fa39c13ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b5720 0x7fa39c13ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 -- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa39c0b44f0 msgr2=0x7fa39c158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa39c0b44f0 0x7fa39c158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 -- 172.21.15.64:0/34626261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c13bf40 con 0x7fa39c141360 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a1d55640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39c141360 0x7fa39c15d530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa38c02bec0 tx=0x7fa38c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa399ffb640 1 -- 172.21.15.64:0/34626261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c035040 con 0x7fa39c141360 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa399ffb640 1 -- 172.21.15.64:0/34626261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c03ac80 con 0x7fa39c141360 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa399ffb640 1 -- 172.21.15.64:0/34626261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c046480 con 0x7fa39c141360 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c1f5570 con 0x7fa39c141360 2024-03-07T07:19:45.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.024+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c1f5e10 con 0x7fa39c141360 2024-03-07T07:19:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.026+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c062c90 con 0x7fa39c141360 2024-03-07T07:19:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.026+0000 7fa399ffb640 1 -- 172.21.15.64:0/34626261 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa38c038020 con 0x7fa39c141360 2024-03-07T07:19:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.026+0000 7fa399ffb640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa38806b4d0 0x7fa38806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.026+0000 7fa399ffb640 1 -- 172.21.15.64:0/34626261 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa38c0ac1f0 con 0x7fa39c141360 2024-03-07T07:19:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.026+0000 7fa3a1554640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa38806b4d0 0x7fa38806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.027+0000 7fa3a1554640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa38806b4d0 0x7fa38806d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa394009bd0 tx=0x7fa39400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.031+0000 7fa399ffb640 1 -- 172.21.15.64:0/34626261 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa39c13bf40 con 0x7fa39c141360 2024-03-07T07:19:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.210+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa39c0b34d0 con 0x7fa39c141360 2024-03-07T07:19:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.210+0000 7fa399ffb640 1 -- 172.21.15.64:0/34626261 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa38c07a9a0 con 0x7fa39c141360 2024-03-07T07:19:45.211 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa38806b4d0 msgr2=0x7fa38806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa38806b4d0 0x7fa38806d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa394009bd0 tx=0x7fa39400b040 comp rx=0 tx=0).stop 2024-03-07T07:19:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39c141360 msgr2=0x7fa39c15d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39c141360 0x7fa39c15d530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa38c02bec0 tx=0x7fa38c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 shutdown_connections 2024-03-07T07:19:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa38806b4d0 0x7fa38806d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa39c141360 0x7fa39c15d530 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa39c0b5720 0x7fa39c13ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 --2- 172.21.15.64:0/34626261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa39c0b44f0 0x7fa39c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 >> 172.21.15.64:0/34626261 conn(0x7fa39c0705e0 msgr2=0x7fa39c0b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:45.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 shutdown_connections 2024-03-07T07:19:45.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.213+0000 7fa3a37b8640 1 -- 172.21.15.64:0/34626261 wait complete. 2024-03-07T07:19:45.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:45.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.328+0000 7f9b96434640 1 Processor -- start 2024-03-07T07:19:45.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b96434640 1 -- start start 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b96434640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9009b0e0 0x7f9b900b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b96434640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9006d280 con 0x7f9b900b1040 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b96434640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9006c400 con 0x7f9b90161990 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b96434640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900b1770 con 0x7f9b9009b0e0 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b949d1640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9b900b1040 0x7f9b900cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46024/0 (socket says 172.21.15.64:46024) 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b949d1640 1 -- 172.21.15.64:0/2867813134 learned_addr learned my addr 172.21.15.64:0/2867813134 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.329+0000 7f9b8f7fe640 1 --2- 172.21.15.64:0/2867813134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9009b0e0 0x7f9b900b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.330+0000 7f9b8effd640 1 -- 172.21.15.64:0/2867813134 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787916376 0 0) 0x7f9b9006d280 con 0x7f9b900b1040 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.330+0000 7f9b8effd640 1 -- 172.21.15.64:0/2867813134 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b78003520 con 0x7f9b900b1040 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.330+0000 7f9b8effd640 1 -- 172.21.15.64:0/2867813134 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635512229 0 0) 0x7f9b9006c400 con 0x7f9b90161990 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.330+0000 7f9b8f7fe640 1 -- 172.21.15.64:0/2867813134 >> v1:172.21.15.120:6789/0 conn(0x7f9b90161990 legacy=0x7f9b9009a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.330+0000 7f9b8f7fe640 1 -- 172.21.15.64:0/2867813134 >> v1:172.21.15.64:6789/0 conn(0x7f9b900b1040 legacy=0x7f9b900cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.330+0000 7f9b8f7fe640 1 -- 172.21.15.64:0/2867813134 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9006c6e0 con 0x7f9b9009b0e0 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b8f7fe640 1 --2- 172.21.15.64:0/2867813134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9009b0e0 0x7f9b900b46b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097d0 tx=0x7f9b8002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef87ad6dab151a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b8effd640 1 -- 172.21.15.64:0/2867813134 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80035020 con 0x7f9b9009b0e0 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b8effd640 1 -- 172.21.15.64:0/2867813134 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80031800 con 0x7f9b9009b0e0 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b8effd640 1 -- 172.21.15.64:0/2867813134 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80031bb0 con 0x7f9b9009b0e0 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b96434640 1 -- 172.21.15.64:0/2867813134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9009b0e0 msgr2=0x7f9b900b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b96434640 1 --2- 172.21.15.64:0/2867813134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9009b0e0 0x7f9b900b46b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097d0 tx=0x7f9b8002b8a0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b96434640 1 -- 172.21.15.64:0/2867813134 shutdown_connections 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b96434640 1 --2- 172.21.15.64:0/2867813134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9009b0e0 0x7f9b900b46b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b96434640 1 -- 172.21.15.64:0/2867813134 >> 172.21.15.64:0/2867813134 conn(0x7f9b900b3b90 msgr2=0x7f9b900b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b96434640 1 -- 172.21.15.64:0/2867813134 shutdown_connections 2024-03-07T07:19:45.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.331+0000 7f9b96434640 1 -- 172.21.15.64:0/2867813134 wait complete. 2024-03-07T07:19:45.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.332+0000 7f9b96434640 1 Processor -- start 2024-03-07T07:19:45.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b96434640 1 -- start start 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b96434640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9009b0e0 0x7f9b900b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b96434640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b900b1040 0x7f9b9009e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b96434640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b90161990 0x7f9b9009eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b96434640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900d0770 con 0x7f9b90161990 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b96434640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900d05f0 con 0x7f9b9009b0e0 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b96434640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900d08f0 con 0x7f9b900b1040 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b949d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b90161990 0x7f9b9009eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b949d1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b90161990 0x7f9b9009eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59262/0 (socket says 172.21.15.64:59262) 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.333+0000 7f9b949d1640 1 -- 172.21.15.64:0/1813339566 learned_addr learned my addr 172.21.15.64:0/1813339566 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b8ffff640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9009b0e0 0x7f9b900b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b8f7fe640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b900b1040 0x7f9b9009e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b949d1640 1 -- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b900b1040 msgr2=0x7f9b9009e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b949d1640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b900b1040 0x7f9b9009e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b949d1640 1 -- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9009b0e0 msgr2=0x7f9b900b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b949d1640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9009b0e0 0x7f9b900b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b949d1640 1 -- 172.21.15.64:0/1813339566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900b6790 con 0x7f9b90161990 2024-03-07T07:19:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b949d1640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b90161990 0x7f9b9009eb10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800cb10 tx=0x7f9b8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b8cff9640 1 -- 172.21.15.64:0/1813339566 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88017070 con 0x7f9b90161990 2024-03-07T07:19:45.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b8cff9640 1 -- 172.21.15.64:0/1813339566 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8800ac90 con 0x7f9b90161990 2024-03-07T07:19:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b8cff9640 1 -- 172.21.15.64:0/1813339566 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88012640 con 0x7f9b90161990 2024-03-07T07:19:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b900b6a70 con 0x7f9b90161990 2024-03-07T07:19:45.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.334+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b900b7030 con 0x7f9b90161990 2024-03-07T07:19:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.335+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b900b7340 con 0x7f9b90161990 2024-03-07T07:19:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.336+0000 7f9b8cff9640 1 -- 172.21.15.64:0/1813339566 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9b8801a070 con 0x7f9b90161990 2024-03-07T07:19:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.336+0000 7f9b8cff9640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.336+0000 7f9b8cff9640 1 -- 172.21.15.64:0/1813339566 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b88089f30 con 0x7f9b90161990 2024-03-07T07:19:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.336+0000 7f9b8ffff640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.337+0000 7f9b8ffff640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000f60 tx=0x7f9b84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.344+0000 7f9b8cff9640 1 -- 172.21.15.64:0/1813339566 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b880586e0 con 0x7f9b90161990 2024-03-07T07:19:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.528+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b90062c90 con 0x7f9b90161990 2024-03-07T07:19:45.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.528+0000 7f9b8cff9640 1 -- 172.21.15.64:0/1813339566 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b8805c780 con 0x7f9b90161990 2024-03-07T07:19:45.528 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7c06b4d0 msgr2=0x7f9b7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b84000f60 tx=0x7f9b84009040 comp rx=0 tx=0).stop 2024-03-07T07:19:45.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b90161990 msgr2=0x7f9b9009eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b90161990 0x7f9b9009eb10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800cb10 tx=0x7f9b8800a060 comp rx=0 tx=0).stop 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 shutdown_connections 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b90161990 0x7f9b9009eb10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b900b1040 0x7f9b9009e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 --2- 172.21.15.64:0/1813339566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9009b0e0 0x7f9b900b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 >> 172.21.15.64:0/1813339566 conn(0x7f9b900b3b90 msgr2=0x7f9b9015e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 shutdown_connections 2024-03-07T07:19:45.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.530+0000 7f9b96434640 1 -- 172.21.15.64:0/1813339566 wait complete. 2024-03-07T07:19:45.545 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:45.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.652+0000 7f6e024c7640 1 Processor -- start 2024-03-07T07:19:45.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.652+0000 7f6e024c7640 1 -- start start 2024-03-07T07:19:45.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e024c7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6dfc15dff0 0x7f6dfc0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e024c7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6dfc0b4500 0x7f6dfc0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e024c7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0b34d0 con 0x7f6dfc1629c0 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e024c7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc161ee0 con 0x7f6dfc0b4500 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e024c7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc1621c0 con 0x7f6dfc15dff0 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e00a64640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6dfc0b4500 0x7f6dfc0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e00a64640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6dfc0b4500 0x7f6dfc0b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59270/0 (socket says 172.21.15.64:59270) 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6dfbfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6dfc15dff0 0x7f6dfc0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e00a64640 1 -- 172.21.15.64:0/2747573843 learned_addr learned my addr 172.21.15.64:0/2747573843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6dfb7fe640 1 --1- 172.21.15.64:0/2747573843 >> v1:172.21.15.120:6789/0 conn(0x7f6dfc1629c0 0x7f6dfc0b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:39718/0 (socket says 172.21.15.64:39718) 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e00a64640 1 -- 172.21.15.64:0/2747573843 >> v2:172.21.15.120:3300/0 conn(0x7f6dfc15dff0 msgr2=0x7f6dfc0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e00a64640 1 --2- 172.21.15.64:0/2747573843 >> v2:172.21.15.120:3300/0 conn(0x7f6dfc15dff0 0x7f6dfc0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e00a64640 1 -- 172.21.15.64:0/2747573843 >> v1:172.21.15.120:6789/0 conn(0x7f6dfc1629c0 legacy=0x7f6dfc0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6e00a64640 1 -- 172.21.15.64:0/2747573843 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc1624a0 con 0x7f6dfc0b4500 2024-03-07T07:19:45.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.653+0000 7f6dfbfff640 1 --2- 172.21.15.64:0/2747573843 >> v2:172.21.15.120:3300/0 conn(0x7f6dfc15dff0 0x7f6dfc0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.654+0000 7f6e00a64640 1 --2- 172.21.15.64:0/2747573843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc0b4500 0x7f6dfc0b4a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc073a70 tx=0x7f6df402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bd2cdf58a8d02c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.654+0000 7f6dfaffd640 1 -- 172.21.15.64:0/2747573843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df402cb20 con 0x7f6dfc0b4500 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6dfaffd640 1 -- 172.21.15.64:0/2747573843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df402ccc0 con 0x7f6dfc0b4500 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6dfaffd640 1 -- 172.21.15.64:0/2747573843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4034610 con 0x7f6dfc0b4500 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 -- 172.21.15.64:0/2747573843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc0b4500 msgr2=0x7f6dfc0b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 --2- 172.21.15.64:0/2747573843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc0b4500 0x7f6dfc0b4a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc073a70 tx=0x7f6df402bf80 comp rx=0 tx=0).stop 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 -- 172.21.15.64:0/2747573843 shutdown_connections 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 --2- 172.21.15.64:0/2747573843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc0b4500 0x7f6dfc0b4a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 --2- 172.21.15.64:0/2747573843 >> v2:172.21.15.120:3300/0 conn(0x7f6dfc15dff0 0x7f6dfc0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 -- 172.21.15.64:0/2747573843 >> 172.21.15.64:0/2747573843 conn(0x7f6dfc06b040 msgr2=0x7f6dfc06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:45.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 -- 172.21.15.64:0/2747573843 shutdown_connections 2024-03-07T07:19:45.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.655+0000 7f6e024c7640 1 -- 172.21.15.64:0/2747573843 wait complete. 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.656+0000 7f6e024c7640 1 Processor -- start 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e024c7640 1 -- start start 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e024c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc0b4500 0x7f6dfc157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e024c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc15dff0 0x7f6dfc1f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e024c7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6dfc1629c0 0x7f6dfc15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e024c7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b3140 con 0x7f6dfc15dff0 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e024c7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b2fc0 con 0x7f6dfc0b4500 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e024c7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15e3d0 con 0x7f6dfc1629c0 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc15dff0 0x7f6dfc1f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc15dff0 0x7f6dfc1f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59276/0 (socket says 172.21.15.64:59276) 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6dfb7fe640 1 -- 172.21.15.64:0/679004555 learned_addr learned my addr 172.21.15.64:0/679004555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:45.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6dfbfff640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc0b4500 0x7f6dfc157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.657+0000 7f6e00a64640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6dfc1629c0 0x7f6dfc15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6dfb7fe640 1 -- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6dfc1629c0 msgr2=0x7f6dfc15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6dfb7fe640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6dfc1629c0 0x7f6dfc15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6dfb7fe640 1 -- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc0b4500 msgr2=0x7f6dfc157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6dfb7fe640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc0b4500 0x7f6dfc157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6dfb7fe640 1 -- 172.21.15.64:0/679004555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc1f4c80 con 0x7f6dfc15dff0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6dfb7fe640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc15dff0 0x7f6dfc1f4740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6df000c3c0 tx=0x7f6df000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6df8ff9640 1 -- 172.21.15.64:0/679004555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df00074e0 con 0x7f6dfc15dff0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6df8ff9640 1 -- 172.21.15.64:0/679004555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df0007680 con 0x7f6dfc15dff0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc15ea70 con 0x7f6dfc15dff0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6df8ff9640 1 -- 172.21.15.64:0/679004555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0004a00 con 0x7f6dfc15dff0 2024-03-07T07:19:45.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.658+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc15f030 con 0x7f6dfc15dff0 2024-03-07T07:19:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.659+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc062c90 con 0x7f6dfc15dff0 2024-03-07T07:19:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.660+0000 7f6df8ff9640 1 -- 172.21.15.64:0/679004555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6df001a020 con 0x7f6dfc15dff0 2024-03-07T07:19:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.660+0000 7f6df8ff9640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6de806b500 0x7f6de806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.660+0000 7f6df8ff9640 1 -- 172.21.15.64:0/679004555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6df00897c0 con 0x7f6dfc15dff0 2024-03-07T07:19:45.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.660+0000 7f6dfbfff640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6de806b500 0x7f6de806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.661+0000 7f6dfbfff640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6de806b500 0x7f6de806d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009560 tx=0x7f6dec009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.664+0000 7f6df8ff9640 1 -- 172.21.15.64:0/679004555 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6df0057ff0 con 0x7f6dfc15dff0 2024-03-07T07:19:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.850+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6dfc0b38f0 con 0x7f6dfc15dff0 2024-03-07T07:19:45.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.851+0000 7f6df8ff9640 1 -- 172.21.15.64:0/679004555 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6df005c090 con 0x7f6dfc15dff0 2024-03-07T07:19:45.851 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:45.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6de806b500 msgr2=0x7f6de806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6de806b500 0x7f6de806d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009560 tx=0x7f6dec009190 comp rx=0 tx=0).stop 2024-03-07T07:19:45.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc15dff0 msgr2=0x7f6dfc1f4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc15dff0 0x7f6dfc1f4740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6df000c3c0 tx=0x7f6df000c880 comp rx=0 tx=0).stop 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 shutdown_connections 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6de806b500 0x7f6de806d9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6dfc15dff0 0x7f6dfc1f4740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6dfc1629c0 0x7f6dfc15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 --2- 172.21.15.64:0/679004555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc0b4500 0x7f6dfc157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 >> 172.21.15.64:0/679004555 conn(0x7f6dfc06b040 msgr2=0x7f6dfc0b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 shutdown_connections 2024-03-07T07:19:45.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.854+0000 7f6e024c7640 1 -- 172.21.15.64:0/679004555 wait complete. 2024-03-07T07:19:45.868 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.970+0000 7f3cb9cf6640 1 Processor -- start 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb9cf6640 1 -- start start 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb9cf6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3cb415fbb0 0x7f3cb40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb9cf6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb4072950 0x7f3cb40cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb9cf6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3cb415ff90 0x7f3cb40acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb9cf6640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb4072f60 con 0x7f3cb415fbb0 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb9cf6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb4073490 con 0x7f3cb415ff90 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb9cf6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40731b0 con 0x7f3cb4072950 2024-03-07T07:19:45.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.971+0000 7f3cb37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3cb415fbb0 0x7f3cb40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb2ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb4072950 0x7f3cb40cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb37fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3cb415fbb0 0x7f3cb40cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59290/0 (socket says 172.21.15.64:59290) 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb37fe640 1 -- 172.21.15.64:0/2439271515 learned_addr learned my addr 172.21.15.64:0/2439271515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb3fff640 1 --2- 172.21.15.64:0/2439271515 >> v2:172.21.15.120:3300/0 conn(0x7f3cb415ff90 0x7f3cb40acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb37fe640 1 -- 172.21.15.64:0/2439271515 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb4072950 msgr2=0x7f3cb40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb37fe640 1 --2- 172.21.15.64:0/2439271515 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb4072950 0x7f3cb40cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb37fe640 1 -- 172.21.15.64:0/2439271515 >> v2:172.21.15.120:3300/0 conn(0x7f3cb415ff90 msgr2=0x7f3cb40acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb37fe640 1 --2- 172.21.15.64:0/2439271515 >> v2:172.21.15.120:3300/0 conn(0x7f3cb415ff90 0x7f3cb40acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.972+0000 7f3cb37fe640 1 -- 172.21.15.64:0/2439271515 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4072150 con 0x7f3cb415fbb0 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb37fe640 1 --2- 172.21.15.64:0/2439271515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb415fbb0 0x7f3cb40cf7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3cb406cf10 tx=0x7f3ca402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142bd072c0557096 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb27fc640 1 -- 172.21.15.64:0/2439271515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca402c9e0 con 0x7f3cb415fbb0 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb27fc640 1 -- 172.21.15.64:0/2439271515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca402cb80 con 0x7f3cb415fbb0 2024-03-07T07:19:45.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb27fc640 1 -- 172.21.15.64:0/2439271515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40343f0 con 0x7f3cb415fbb0 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/2439271515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb415fbb0 msgr2=0x7f3cb40cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/2439271515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb415fbb0 0x7f3cb40cf7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3cb406cf10 tx=0x7f3ca402be40 comp rx=0 tx=0).stop 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/2439271515 shutdown_connections 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/2439271515 >> v2:172.21.15.120:3300/0 conn(0x7f3cb415ff90 0x7f3cb40acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/2439271515 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb4072950 0x7f3cb40cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/2439271515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb415fbb0 0x7f3cb40cf7d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/2439271515 >> 172.21.15.64:0/2439271515 conn(0x7f3cb40b3120 msgr2=0x7f3cb40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/2439271515 shutdown_connections 2024-03-07T07:19:45.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.973+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/2439271515 wait complete. 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.975+0000 7f3cb9cf6640 1 Processor -- start 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.975+0000 7f3cb9cf6640 1 -- start start 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.975+0000 7f3cb9cf6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb415fbb0 0x7f3cb4159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.975+0000 7f3cb9cf6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb415ff90 0x7f3cb40c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.975+0000 7f3cb9cf6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40c5fe0 0x7f3cb415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.975+0000 7f3cb9cf6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40b1a00 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb9cf6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4160980 con 0x7f3cb415ff90 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb9cf6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb41603e0 con 0x7f3cb415fbb0 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40c5fe0 0x7f3cb415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40c5fe0 0x7f3cb415dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59304/0 (socket says 172.21.15.64:59304) 2024-03-07T07:19:45.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 -- 172.21.15.64:0/1077030015 learned_addr learned my addr 172.21.15.64:0/1077030015 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb2ffd640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb415ff90 0x7f3cb40c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb37fe640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb415fbb0 0x7f3cb4159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 -- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb415fbb0 msgr2=0x7f3cb4159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb415fbb0 0x7f3cb4159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 -- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb415ff90 msgr2=0x7f3cb40c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb415ff90 0x7f3cb40c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 -- 172.21.15.64:0/1077030015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40c6480 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb3fff640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40c5fe0 0x7f3cb415dd20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00e930 tx=0x7f3cac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3c9bfff640 1 -- 172.21.15.64:0/1077030015 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac017070 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3c9bfff640 1 -- 172.21.15.64:0/1077030015 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac00c980 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb40c6760 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3c9bfff640 1 -- 172.21.15.64:0/1077030015 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00cd70 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.976+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb415a240 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.977+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4062c90 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.978+0000 7f3c9bfff640 1 -- 172.21.15.64:0/1077030015 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3cac01a070 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.978+0000 7f3c9bfff640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3ca006b500 0x7f3ca006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.978+0000 7f3c9bfff640 1 -- 172.21.15.64:0/1077030015 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cac08a490 con 0x7f3cb40c5fe0 2024-03-07T07:19:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.978+0000 7f3cb37fe640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3ca006b500 0x7f3ca006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:45.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.979+0000 7f3cb37fe640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3ca006b500 0x7f3ca006d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4035000 tx=0x7f3ca4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:45.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:45.983+0000 7f3c9bfff640 1 -- 172.21.15.64:0/1077030015 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3cac058e10 con 0x7f3cb40c5fe0 2024-03-07T07:19:46.128 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:19:46.130 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:19:46.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.236+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7f3cb40c6c80 con 0x7f3cb40c5fe0 2024-03-07T07:19:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.383+0000 7f3c9bfff640 1 -- 172.21.15.64:0/1077030015 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v51) v1 ==== 147+0+0 (secure 0 0 0) 0x7f3cac05ceb0 con 0x7f3cb40c5fe0 2024-03-07T07:19:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3ca006b500 msgr2=0x7f3ca006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3ca006b500 0x7f3ca006d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4035000 tx=0x7f3ca4030000 comp rx=0 tx=0).stop 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40c5fe0 msgr2=0x7f3cb415dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40c5fe0 0x7f3cb415dd20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00e930 tx=0x7f3cac00edf0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 shutdown_connections 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3ca006b500 0x7f3ca006d9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cb40c5fe0 0x7f3cb415dd20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb415ff90 0x7f3cb40c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 --2- 172.21.15.64:0/1077030015 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3cb415fbb0 0x7f3cb4159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 >> 172.21.15.64:0/1077030015 conn(0x7f3cb40b3120 msgr2=0x7f3cb4074270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 shutdown_connections 2024-03-07T07:19:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.385+0000 7f3cb9cf6640 1 -- 172.21.15.64:0/1077030015 wait complete. 2024-03-07T07:19:46.396 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-07T07:19:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.501+0000 7fb3e93de640 1 Processor -- start 2024-03-07T07:19:46.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.501+0000 7fb3e93de640 1 -- start start 2024-03-07T07:19:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e93de640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40cea70 0x7fb3e415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e93de640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e4070eb0 con 0x7fb3e415e870 2024-03-07T07:19:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e93de640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e4071190 con 0x7fb3e40d3310 2024-03-07T07:19:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e93de640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40cf8a0 con 0x7fb3e40cea70 2024-03-07T07:19:46.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e2ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb3e415e870 0x7fb3e40cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46036/0 (socket says 172.21.15.64:46036) 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e2ffd640 1 -- 172.21.15.64:0/3768116627 learned_addr learned my addr 172.21.15.64:0/3768116627 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e27fc640 1 --2- 172.21.15.64:0/3768116627 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40cea70 0x7fb3e415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e1ffb640 1 -- 172.21.15.64:0/3768116627 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317338212 0 0) 0x7fb3e4070eb0 con 0x7fb3e415e870 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e1ffb640 1 -- 172.21.15.64:0/3768116627 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d0003520 con 0x7fb3e415e870 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e1ffb640 1 -- 172.21.15.64:0/3768116627 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195870021 0 0) 0x7fb3e4071190 con 0x7fb3e40d3310 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e1ffb640 1 -- 172.21.15.64:0/3768116627 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e4070eb0 con 0x7fb3e40d3310 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e27fc640 1 -- 172.21.15.64:0/3768116627 >> v1:172.21.15.120:6789/0 conn(0x7fb3e40d3310 legacy=0x7fb3e4167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e27fc640 1 -- 172.21.15.64:0/3768116627 >> v1:172.21.15.64:6789/0 conn(0x7fb3e415e870 legacy=0x7fb3e40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.502+0000 7fb3e27fc640 1 -- 172.21.15.64:0/3768116627 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4071470 con 0x7fb3e40cea70 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.503+0000 7fb3e27fc640 1 --2- 172.21.15.64:0/3768116627 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40cea70 0x7fb3e415ec50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40097d0 tx=0x7fb3d402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68fdc94e3760ce7c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.503+0000 7fb3e1ffb640 1 -- 172.21.15.64:0/3768116627 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4035020 con 0x7fb3e40cea70 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.503+0000 7fb3e1ffb640 1 -- 172.21.15.64:0/3768116627 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4031890 con 0x7fb3e40cea70 2024-03-07T07:19:46.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.503+0000 7fb3e1ffb640 1 -- 172.21.15.64:0/3768116627 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4031c60 con 0x7fb3e40cea70 2024-03-07T07:19:46.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.504+0000 7fb3e93de640 1 -- 172.21.15.64:0/3768116627 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40cea70 msgr2=0x7fb3e415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.504+0000 7fb3e93de640 1 --2- 172.21.15.64:0/3768116627 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40cea70 0x7fb3e415ec50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40097d0 tx=0x7fb3d402b930 comp rx=0 tx=0).stop 2024-03-07T07:19:46.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.504+0000 7fb3e93de640 1 -- 172.21.15.64:0/3768116627 shutdown_connections 2024-03-07T07:19:46.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.504+0000 7fb3e93de640 1 --2- 172.21.15.64:0/3768116627 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40cea70 0x7fb3e415ec50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.504+0000 7fb3e93de640 1 -- 172.21.15.64:0/3768116627 >> 172.21.15.64:0/3768116627 conn(0x7fb3e40d11e0 msgr2=0x7fb3e40d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:46.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.504+0000 7fb3e93de640 1 -- 172.21.15.64:0/3768116627 shutdown_connections 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.504+0000 7fb3e93de640 1 -- 172.21.15.64:0/3768116627 wait complete. 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.505+0000 7fb3e93de640 1 Processor -- start 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.505+0000 7fb3e93de640 1 -- start start 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e93de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3e40cea70 0x7fb3e40aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e93de640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40d3310 0x7fb3e40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e93de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3e415e870 0x7fb3e40cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e93de640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4070b60 con 0x7fb3e415e870 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e93de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4071a30 con 0x7fb3e40cea70 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e93de640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e415f190 con 0x7fb3e40d3310 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3e415e870 0x7fb3e40cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e2ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3e40cea70 0x7fb3e40aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3e415e870 0x7fb3e40cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59320/0 (socket says 172.21.15.64:59320) 2024-03-07T07:19:46.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e37fe640 1 -- 172.21.15.64:0/2126815312 learned_addr learned my addr 172.21.15.64:0/2126815312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.506+0000 7fb3e27fc640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40d3310 0x7fb3e40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e37fe640 1 -- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40d3310 msgr2=0x7fb3e40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e37fe640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40d3310 0x7fb3e40cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e37fe640 1 -- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3e40cea70 msgr2=0x7fb3e40aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e37fe640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3e40cea70 0x7fb3e40aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e37fe640 1 -- 172.21.15.64:0/2126815312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e415b560 con 0x7fb3e415e870 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e2ffd640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3e40cea70 0x7fb3e40aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e27fc640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40d3310 0x7fb3e40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e37fe640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3e415e870 0x7fb3e40cd750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00c7c0 tx=0x7fb3dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3db7fe640 1 -- 172.21.15.64:0/2126815312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc003b70 con 0x7fb3e415e870 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3db7fe640 1 -- 172.21.15.64:0/2126815312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc003d10 con 0x7fb3e415e870 2024-03-07T07:19:46.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3db7fe640 1 -- 172.21.15.64:0/2126815312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00aa50 con 0x7fb3e415e870 2024-03-07T07:19:46.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e415ad60 con 0x7fb3e415e870 2024-03-07T07:19:46.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.507+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e4148710 con 0x7fb3e415e870 2024-03-07T07:19:46.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.508+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4070400 con 0x7fb3e415e870 2024-03-07T07:19:46.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.508+0000 7fb3db7fe640 1 -- 172.21.15.64:0/2126815312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb3dc00abf0 con 0x7fb3e415e870 2024-03-07T07:19:46.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.509+0000 7fb3db7fe640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb3c406b4d0 0x7fb3c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:46.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.509+0000 7fb3db7fe640 1 -- 172.21.15.64:0/2126815312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3dc08a090 con 0x7fb3e415e870 2024-03-07T07:19:46.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.509+0000 7fb3e2ffd640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb3c406b4d0 0x7fb3c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:46.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.509+0000 7fb3e2ffd640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb3c406b4d0 0x7fb3c406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000f60 tx=0x7fb3cc007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:46.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.513+0000 7fb3db7fe640 1 -- 172.21.15.64:0/2126815312 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb3e4070400 con 0x7fb3e415e870 2024-03-07T07:19:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.698+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb3e4062c90 con 0x7fb3e415e870 2024-03-07T07:19:46.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.700+0000 7fb3db7fe640 1 -- 172.21.15.64:0/2126815312 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3e415b560 con 0x7fb3e415e870 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.702+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb3c406b4d0 msgr2=0x7fb3c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.702+0000 7fb3e93de640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb3c406b4d0 0x7fb3c406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000f60 tx=0x7fb3cc007d70 comp rx=0 tx=0).stop 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3e415e870 msgr2=0x7fb3e40cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3e415e870 0x7fb3e40cd750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00c7c0 tx=0x7fb3dc00cc80 comp rx=0 tx=0).stop 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 shutdown_connections 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb3c406b4d0 0x7fb3c406d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3e415e870 0x7fb3e40cd750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3e40d3310 0x7fb3e40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 --2- 172.21.15.64:0/2126815312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3e40cea70 0x7fb3e40aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:46.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 >> 172.21.15.64:0/2126815312 conn(0x7fb3e40d11e0 msgr2=0x7fb3e40cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:46.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 shutdown_connections 2024-03-07T07:19:46.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:46.703+0000 7fb3e93de640 1 -- 172.21.15.64:0/2126815312 wait complete. 2024-03-07T07:19:46.713 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-07T07:19:46.714 DEBUG:teuthology.orchestra.run.smithi064:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T07:19:46.766 DEBUG:teuthology.orchestra.run.smithi120:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T07:19:46.810 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-07T07:19:46.810 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:46.810 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:46.826 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:46.826 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:46.881 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:46.881 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:46.935 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:47.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.076+0000 7f46d9234640 1 Processor -- start 2024-03-07T07:19:47.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.076+0000 7f46d9234640 1 -- start start 2024-03-07T07:19:47.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.077+0000 7f46d9234640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f46d4071410 0x7f46d4071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.077+0000 7f46d9234640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d4162c80 con 0x7f46d4144540 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.077+0000 7f46d9234640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d40b4bd0 con 0x7f46d40b5dd0 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.077+0000 7f46d9234640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d4141700 con 0x7f46d4071410 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.077+0000 7f46d359e640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f46d4071410 0x7f46d4071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.077+0000 7f46d259c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f46d4144540 0x7f46d4162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46040/0 (socket says 172.21.15.64:46040) 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.077+0000 7f46d259c640 1 -- 172.21.15.64:0/1396503540 learned_addr learned my addr 172.21.15.64:0/1396503540 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.078+0000 7f46d359e640 1 -- 172.21.15.64:0/1396503540 >> v1:172.21.15.120:6789/0 conn(0x7f46d40b5dd0 legacy=0x7f46d40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.078+0000 7f46d359e640 1 -- 172.21.15.64:0/1396503540 >> v1:172.21.15.64:6789/0 conn(0x7f46d4144540 legacy=0x7f46d4162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.078+0000 7f46d359e640 1 -- 172.21.15.64:0/1396503540 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40b4eb0 con 0x7f46d4071410 2024-03-07T07:19:47.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.078+0000 7f46d359e640 1 --2- 172.21.15.64:0/1396503540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4071410 0x7f46d4071880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009d90 tx=0x7f46c402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3efc3fa549aedbc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.078+0000 7f46d1d9b640 1 -- 172.21.15.64:0/1396503540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4039070 con 0x7f46d4071410 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.078+0000 7f46d1d9b640 1 -- 172.21.15.64:0/1396503540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c402cc60 con 0x7f46d4071410 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.078+0000 7f46d1d9b640 1 -- 172.21.15.64:0/1396503540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4034660 con 0x7f46d4071410 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.079+0000 7f46d9234640 1 -- 172.21.15.64:0/1396503540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4071410 msgr2=0x7f46d4071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.079+0000 7f46d9234640 1 --2- 172.21.15.64:0/1396503540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4071410 0x7f46d4071880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009d90 tx=0x7f46c402beb0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.079+0000 7f46d9234640 1 -- 172.21.15.64:0/1396503540 shutdown_connections 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.079+0000 7f46d9234640 1 --2- 172.21.15.64:0/1396503540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4071410 0x7f46d4071880 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.079+0000 7f46d9234640 1 -- 172.21.15.64:0/1396503540 >> 172.21.15.64:0/1396503540 conn(0x7f46d40d0e70 msgr2=0x7f46d40d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.079+0000 7f46d9234640 1 -- 172.21.15.64:0/1396503540 shutdown_connections 2024-03-07T07:19:47.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.079+0000 7f46d9234640 1 -- 172.21.15.64:0/1396503540 wait complete. 2024-03-07T07:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.080+0000 7f46d9234640 1 Processor -- start 2024-03-07T07:19:47.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.080+0000 7f46d9234640 1 -- start start 2024-03-07T07:19:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d9234640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d4071410 0x7f46d413cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d9234640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d40b5dd0 0x7f46d415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d9234640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4144540 0x7f46d415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d9234640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4161e00 con 0x7f46d40b5dd0 2024-03-07T07:19:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d9234640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4071dc0 con 0x7f46d4071410 2024-03-07T07:19:47.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d9234640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4162050 con 0x7f46d4144540 2024-03-07T07:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d40b5dd0 0x7f46d415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d40b5dd0 0x7f46d415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59330/0 (socket says 172.21.15.64:59330) 2024-03-07T07:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d259c640 1 -- 172.21.15.64:0/2097116426 learned_addr learned my addr 172.21.15.64:0/2097116426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d2d9d640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d4071410 0x7f46d413cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.081+0000 7f46d359e640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4144540 0x7f46d415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d259c640 1 -- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4144540 msgr2=0x7f46d415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d259c640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4144540 0x7f46d415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d259c640 1 -- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d4071410 msgr2=0x7f46d413cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d259c640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d4071410 0x7f46d413cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d259c640 1 -- 172.21.15.64:0/2097116426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d413d2b0 con 0x7f46d40b5dd0 2024-03-07T07:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d259c640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d40b5dd0 0x7f46d415d810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c480 tx=0x7f46bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46cb7fe640 1 -- 172.21.15.64:0/2097116426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc017070 con 0x7f46d40b5dd0 2024-03-07T07:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46cb7fe640 1 -- 172.21.15.64:0/2097116426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc005ce0 con 0x7f46d40b5dd0 2024-03-07T07:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d4157060 con 0x7f46d40b5dd0 2024-03-07T07:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46cb7fe640 1 -- 172.21.15.64:0/2097116426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0072f0 con 0x7f46d40b5dd0 2024-03-07T07:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.082+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d4157620 con 0x7f46d40b5dd0 2024-03-07T07:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.083+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d4062c90 con 0x7f46d40b5dd0 2024-03-07T07:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.083+0000 7f46cb7fe640 1 -- 172.21.15.64:0/2097116426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f46bc01a070 con 0x7f46d40b5dd0 2024-03-07T07:19:47.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.084+0000 7f46cb7fe640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46b406b4d0 0x7f46b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.084+0000 7f46cb7fe640 1 -- 172.21.15.64:0/2097116426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46bc0893e0 con 0x7f46d40b5dd0 2024-03-07T07:19:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.084+0000 7f46d2d9d640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46b406b4d0 0x7f46b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.084+0000 7f46d2d9d640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46b406b4d0 0x7f46b406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f46cc001780 tx=0x7f46cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.088+0000 7f46cb7fe640 1 -- 172.21.15.64:0/2097116426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46d413d2b0 con 0x7f46d40b5dd0 2024-03-07T07:19:47.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.271+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46d40b8a10 con 0x7f46d40b5dd0 2024-03-07T07:19:47.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.272+0000 7f46cb7fe640 1 -- 172.21.15.64:0/2097116426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f46bc057ca0 con 0x7f46d40b5dd0 2024-03-07T07:19:47.272 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:47.272 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46b406b4d0 msgr2=0x7f46b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46b406b4d0 0x7f46b406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f46cc001780 tx=0x7f46cc009040 comp rx=0 tx=0).stop 2024-03-07T07:19:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d40b5dd0 msgr2=0x7f46d415d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d40b5dd0 0x7f46d415d810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c480 tx=0x7f46bc00c940 comp rx=0 tx=0).stop 2024-03-07T07:19:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 shutdown_connections 2024-03-07T07:19:47.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46b406b4d0 0x7f46b406d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d40b5dd0 0x7f46d415d810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4144540 0x7f46d415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 --2- 172.21.15.64:0/2097116426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d4071410 0x7f46d413cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 >> 172.21.15.64:0/2097116426 conn(0x7f46d40d0e70 msgr2=0x7f46d40d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 shutdown_connections 2024-03-07T07:19:47.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.274+0000 7f46d9234640 1 -- 172.21.15.64:0/2097116426 wait complete. 2024-03-07T07:19:47.285 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:47.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.390+0000 7f7cb981e640 1 Processor -- start 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.390+0000 7f7cb981e640 1 -- start start 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb981e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb41625c0 0x7f7cb4071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb981e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb4071ff0 con 0x7f7cb40b7200 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb981e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40d23b0 con 0x7f7cb415e350 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb981e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40d2160 con 0x7f7cb41625c0 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb37fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7cb40b7200 0x7f7cb40717c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46042/0 (socket says 172.21.15.64:46042) 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb37fe640 1 -- 172.21.15.64:0/244979204 learned_addr learned my addr 172.21.15.64:0/244979204 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb27fc640 1 --2- 172.21.15.64:0/244979204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb41625c0 0x7f7cb4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb1ffb640 1 -- 172.21.15.64:0/244979204 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458586343 0 0) 0x7f7cb4071ff0 con 0x7f7cb40b7200 2024-03-07T07:19:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.391+0000 7f7cb1ffb640 1 -- 172.21.15.64:0/244979204 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca0003520 con 0x7f7cb40b7200 2024-03-07T07:19:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.392+0000 7f7cb1ffb640 1 -- 172.21.15.64:0/244979204 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325134133 0 0) 0x7f7cb40d23b0 con 0x7f7cb415e350 2024-03-07T07:19:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.392+0000 7f7cb1ffb640 1 -- 172.21.15.64:0/244979204 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb4071ff0 con 0x7f7cb415e350 2024-03-07T07:19:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.392+0000 7f7cb27fc640 1 -- 172.21.15.64:0/244979204 >> v1:172.21.15.120:6789/0 conn(0x7f7cb415e350 legacy=0x7f7cb4161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.392+0000 7f7cb27fc640 1 -- 172.21.15.64:0/244979204 >> v1:172.21.15.64:6789/0 conn(0x7f7cb40b7200 legacy=0x7f7cb40717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.392+0000 7f7cb27fc640 1 -- 172.21.15.64:0/244979204 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40d2690 con 0x7f7cb41625c0 2024-03-07T07:19:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.392+0000 7f7cb27fc640 1 --2- 172.21.15.64:0/244979204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb41625c0 0x7f7cb4071280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0097d0 tx=0x7f7c9c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e3cb3b5adbff08 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.392+0000 7f7cb1ffb640 1 -- 172.21.15.64:0/244979204 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c035020 con 0x7f7cb41625c0 2024-03-07T07:19:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb1ffb640 1 -- 172.21.15.64:0/244979204 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c031890 con 0x7f7cb41625c0 2024-03-07T07:19:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb1ffb640 1 -- 172.21.15.64:0/244979204 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c031c40 con 0x7f7cb41625c0 2024-03-07T07:19:47.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb981e640 1 -- 172.21.15.64:0/244979204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb41625c0 msgr2=0x7f7cb4071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb981e640 1 --2- 172.21.15.64:0/244979204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb41625c0 0x7f7cb4071280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0097d0 tx=0x7f7c9c02b930 comp rx=0 tx=0).stop 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb981e640 1 -- 172.21.15.64:0/244979204 shutdown_connections 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb981e640 1 --2- 172.21.15.64:0/244979204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb41625c0 0x7f7cb4071280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb981e640 1 -- 172.21.15.64:0/244979204 >> 172.21.15.64:0/244979204 conn(0x7f7cb40d0e70 msgr2=0x7f7cb40d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb981e640 1 -- 172.21.15.64:0/244979204 shutdown_connections 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.393+0000 7f7cb981e640 1 -- 172.21.15.64:0/244979204 wait complete. 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.394+0000 7f7cb981e640 1 Processor -- start 2024-03-07T07:19:47.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.394+0000 7f7cb981e640 1 -- start start 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb981e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cb40b7200 0x7f7cb415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb981e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb415e350 0x7f7cb4154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb981e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb41625c0 0x7f7cb4144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb981e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb41691e0 con 0x7f7cb41625c0 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb981e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415e730 con 0x7f7cb40b7200 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb981e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4169360 con 0x7f7cb415e350 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb41625c0 0x7f7cb4144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb41625c0 0x7f7cb4144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59346/0 (socket says 172.21.15.64:59346) 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb37fe640 1 -- 172.21.15.64:0/2589317668 learned_addr learned my addr 172.21.15.64:0/2589317668 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb2ffd640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cb40b7200 0x7f7cb415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.395+0000 7f7cb27fc640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb415e350 0x7f7cb4154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb2ffd640 1 -- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb415e350 msgr2=0x7f7cb4154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb2ffd640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb415e350 0x7f7cb4154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb2ffd640 1 -- 172.21.15.64:0/2589317668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb41625c0 msgr2=0x7f7cb4144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb2ffd640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb41625c0 0x7f7cb4144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb2ffd640 1 -- 172.21.15.64:0/2589317668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb415f850 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb37fe640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb41625c0 0x7f7cb4144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb2ffd640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cb40b7200 0x7f7cb415f250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c730 tx=0x7f7ca400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cabfff640 1 -- 172.21.15.64:0/2589317668 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4003b70 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cabfff640 1 -- 172.21.15.64:0/2589317668 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4003d30 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cabfff640 1 -- 172.21.15.64:0/2589317668 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4005310 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb415fb30 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.396+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb41600f0 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.397+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb4062c90 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.397+0000 7f7cabfff640 1 -- 172.21.15.64:0/2589317668 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7ca40056c0 con 0x7f7cb40b7200 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.398+0000 7f7cabfff640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c9406b4d0 0x7f7c9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.398+0000 7f7cabfff640 1 -- 172.21.15.64:0/2589317668 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ca408a140 con 0x7f7cb40b7200 2024-03-07T07:19:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.398+0000 7f7cb27fc640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c9406b4d0 0x7f7c9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.398+0000 7f7cb27fc640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c9406b4d0 0x7f7c9406d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c036000 tx=0x7f7c9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.403+0000 7f7cabfff640 1 -- 172.21.15.64:0/2589317668 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ca4058970 con 0x7f7cb40b7200 2024-03-07T07:19:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.582+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cb40d1d10 con 0x7f7cb40b7200 2024-03-07T07:19:47.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.582+0000 7f7cabfff640 1 -- 172.21.15.64:0/2589317668 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ca405ca10 con 0x7f7cb40b7200 2024-03-07T07:19:47.582 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:47.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c9406b4d0 msgr2=0x7f7c9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c9406b4d0 0x7f7c9406d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c036000 tx=0x7f7c9c008030 comp rx=0 tx=0).stop 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cb40b7200 msgr2=0x7f7cb415f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cb40b7200 0x7f7cb415f250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c730 tx=0x7f7ca400cbf0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 shutdown_connections 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7c9406b4d0 0x7f7c9406d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cb41625c0 0x7f7cb4144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7cb415e350 0x7f7cb4154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 --2- 172.21.15.64:0/2589317668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cb40b7200 0x7f7cb415f250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 >> 172.21.15.64:0/2589317668 conn(0x7f7cb40d0e70 msgr2=0x7f7cb40c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:47.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 shutdown_connections 2024-03-07T07:19:47.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.584+0000 7f7cb981e640 1 -- 172.21.15.64:0/2589317668 wait complete. 2024-03-07T07:19:47.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.699+0000 7f735b71d640 1 Processor -- start 2024-03-07T07:19:47.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.699+0000 7f735b71d640 1 -- start start 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f735b71d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f7354165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f735b71d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735406d740 con 0x7f73540b3ed0 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f735b71d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735406da20 con 0x7f735415b370 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f735b71d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735406dd00 con 0x7f735415be60 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f7358cb8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f73540b3ed0 0x7f735415b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46056/0 (socket says 172.21.15.64:46056) 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f7358cb8640 1 -- 172.21.15.64:0/3611310512 learned_addr learned my addr 172.21.15.64:0/3611310512 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f7359cba640 1 --2- 172.21.15.64:0/3611310512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f7354165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f7353fff640 1 -- 172.21.15.64:0/3611310512 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870861172 0 0) 0x7f735406d740 con 0x7f73540b3ed0 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.700+0000 7f7353fff640 1 -- 172.21.15.64:0/3611310512 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f733c003520 con 0x7f73540b3ed0 2024-03-07T07:19:47.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.701+0000 7f7353fff640 1 -- 172.21.15.64:0/3611310512 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665105155 0 0) 0x7f735406da20 con 0x7f735415b370 2024-03-07T07:19:47.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.701+0000 7f7353fff640 1 -- 172.21.15.64:0/3611310512 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f735406d740 con 0x7f735415b370 2024-03-07T07:19:47.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.701+0000 7f7359cba640 1 -- 172.21.15.64:0/3611310512 >> v1:172.21.15.120:6789/0 conn(0x7f735415b370 legacy=0x7f73540cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.701+0000 7f7359cba640 1 -- 172.21.15.64:0/3611310512 >> v1:172.21.15.64:6789/0 conn(0x7f73540b3ed0 legacy=0x7f735415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.701+0000 7f7359cba640 1 -- 172.21.15.64:0/3611310512 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735406dfe0 con 0x7f735415be60 2024-03-07T07:19:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.701+0000 7f7359cba640 1 --2- 172.21.15.64:0/3611310512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f7354165930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7344009dc0 tx=0x7f734402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14614609aaf571cd server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f7353fff640 1 -- 172.21.15.64:0/3611310512 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344039070 con 0x7f735415be60 2024-03-07T07:19:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f7353fff640 1 -- 172.21.15.64:0/3611310512 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734402ccd0 con 0x7f735415be60 2024-03-07T07:19:47.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f7353fff640 1 -- 172.21.15.64:0/3611310512 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344034660 con 0x7f735415be60 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f735b71d640 1 -- 172.21.15.64:0/3611310512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 msgr2=0x7f7354165930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f735b71d640 1 --2- 172.21.15.64:0/3611310512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f7354165930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7344009dc0 tx=0x7f734402bf20 comp rx=0 tx=0).stop 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f735b71d640 1 -- 172.21.15.64:0/3611310512 shutdown_connections 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f735b71d640 1 --2- 172.21.15.64:0/3611310512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f7354165930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f735b71d640 1 -- 172.21.15.64:0/3611310512 >> 172.21.15.64:0/3611310512 conn(0x7f73540b1d30 msgr2=0x7f73540b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f735b71d640 1 -- 172.21.15.64:0/3611310512 shutdown_connections 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.702+0000 7f735b71d640 1 -- 172.21.15.64:0/3611310512 wait complete. 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.703+0000 7f735b71d640 1 Processor -- start 2024-03-07T07:19:47.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.703+0000 7f735b71d640 1 -- start start 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f735b71d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73540b3ed0 0x7f735415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f735b71d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735415b370 0x7f7354141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f735b71d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f73541e3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f735b71d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540cf7e0 con 0x7f735415b370 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f735b71d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540cf660 con 0x7f73540b3ed0 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f735b71d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735406d180 con 0x7f735415be60 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735415b370 0x7f7354141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735415b370 0x7f7354141610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59350/0 (socket says 172.21.15.64:59350) 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 -- 172.21.15.64:0/1357189426 learned_addr learned my addr 172.21.15.64:0/1357189426 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f73594b9640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73540b3ed0 0x7f735415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7359cba640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f73541e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 -- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 msgr2=0x7f73541e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f73541e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 -- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73540b3ed0 msgr2=0x7f735415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73540b3ed0 0x7f735415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f7358cb8640 1 -- 172.21.15.64:0/1357189426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73541585f0 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.704+0000 7f73594b9640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73540b3ed0 0x7f735415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.705+0000 7f7358cb8640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735415b370 0x7f7354141610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f734800c530 tx=0x7f734800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.705+0000 7f7351ffb640 1 -- 172.21.15.64:0/1357189426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348017070 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.705+0000 7f7351ffb640 1 -- 172.21.15.64:0/1357189426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7348005d00 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.705+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73541588d0 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.705+0000 7f7351ffb640 1 -- 172.21.15.64:0/1357189426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480072b0 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.705+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7354158e90 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.706+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7354159090 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.706+0000 7f7351ffb640 1 -- 172.21.15.64:0/1357189426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f734801a070 con 0x7f735415b370 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.707+0000 7f7351ffb640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f734006b4d0 0x7f734006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:47.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.707+0000 7f7351ffb640 1 -- 172.21.15.64:0/1357189426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73480894f0 con 0x7f735415b370 2024-03-07T07:19:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.707+0000 7f73594b9640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f734006b4d0 0x7f734006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:47.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.707+0000 7f73594b9640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f734006b4d0 0x7f734006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f734c007fd0 tx=0x7f734c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:47.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.711+0000 7f7351ffb640 1 -- 172.21.15.64:0/1357189426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7354159090 con 0x7f735415b370 2024-03-07T07:19:47.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.892+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7354062c90 con 0x7f735415b370 2024-03-07T07:19:47.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.892+0000 7f7351ffb640 1 -- 172.21.15.64:0/1357189426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7348057e70 con 0x7f735415b370 2024-03-07T07:19:47.892 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:47.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f734006b4d0 msgr2=0x7f734006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f734006b4d0 0x7f734006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f734c007fd0 tx=0x7f734c009040 comp rx=0 tx=0).stop 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735415b370 msgr2=0x7f7354141610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735415b370 0x7f7354141610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f734800c530 tx=0x7f734800c9f0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 shutdown_connections 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f734006b4d0 0x7f734006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f735415b370 0x7f7354141610 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f735415be60 0x7f73541e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 --2- 172.21.15.64:0/1357189426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73540b3ed0 0x7f735415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 >> 172.21.15.64:0/1357189426 conn(0x7f73540b1d30 msgr2=0x7f7354072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 shutdown_connections 2024-03-07T07:19:47.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:47.894+0000 7f735b71d640 1 -- 172.21.15.64:0/1357189426 wait complete. 2024-03-07T07:19:47.905 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:48.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.009+0000 7f77998bf640 1 Processor -- start 2024-03-07T07:19:48.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.009+0000 7f77998bf640 1 -- start start 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.009+0000 7f77998bf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cf610 0x7f77940cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.009+0000 7f77998bf640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f77940cda20 0x7f77940719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.009+0000 7f77998bf640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779406e150 con 0x7f7794145670 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.009+0000 7f77998bf640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77940cfa70 con 0x7f77940cda20 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.009+0000 7f77998bf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779406ed60 con 0x7f77940cf610 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f7792ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7794145670 0x7f779415cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46066/0 (socket says 172.21.15.64:46066) 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f7792ffd640 1 -- 172.21.15.64:0/548347000 learned_addr learned my addr 172.21.15.64:0/548347000 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f77927fc640 1 --2- 172.21.15.64:0/548347000 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cf610 0x7f77940cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f77937fe640 1 --2- 172.21.15.64:0/548347000 >> v2:172.21.15.120:3300/0 conn(0x7f77940cda20 0x7f77940719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f7791ffb640 1 -- 172.21.15.64:0/548347000 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634153985 0 0) 0x7f779406e150 con 0x7f7794145670 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f7791ffb640 1 -- 172.21.15.64:0/548347000 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7780003520 con 0x7f7794145670 2024-03-07T07:19:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f77927fc640 1 -- 172.21.15.64:0/548347000 >> v2:172.21.15.120:3300/0 conn(0x7f77940cda20 msgr2=0x7f77940719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f77927fc640 1 --2- 172.21.15.64:0/548347000 >> v2:172.21.15.120:3300/0 conn(0x7f77940cda20 0x7f77940719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f77927fc640 1 -- 172.21.15.64:0/548347000 >> v1:172.21.15.64:6789/0 conn(0x7f7794145670 legacy=0x7f779415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f77927fc640 1 -- 172.21.15.64:0/548347000 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779406e430 con 0x7f77940cf610 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.010+0000 7f77937fe640 1 --2- 172.21.15.64:0/548347000 >> v2:172.21.15.120:3300/0 conn(0x7f77940cda20 0x7f77940719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77927fc640 1 --2- 172.21.15.64:0/548347000 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cf610 0x7f77940cd4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f777c0097a0 tx=0x7f777c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9879ebca7a74fc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f7791ffb640 1 -- 172.21.15.64:0/548347000 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0315a0 con 0x7f77940cf610 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f7791ffb640 1 -- 172.21.15.64:0/548347000 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c037070 con 0x7f77940cf610 2024-03-07T07:19:48.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f7791ffb640 1 -- 172.21.15.64:0/548347000 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c031940 con 0x7f77940cf610 2024-03-07T07:19:48.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 -- 172.21.15.64:0/548347000 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cf610 msgr2=0x7f77940cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 --2- 172.21.15.64:0/548347000 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cf610 0x7f77940cd4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f777c0097a0 tx=0x7f777c02b840 comp rx=0 tx=0).stop 2024-03-07T07:19:48.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 -- 172.21.15.64:0/548347000 shutdown_connections 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 --2- 172.21.15.64:0/548347000 >> v2:172.21.15.120:3300/0 conn(0x7f77940cda20 0x7f77940719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 --2- 172.21.15.64:0/548347000 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cf610 0x7f77940cd4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 -- 172.21.15.64:0/548347000 >> 172.21.15.64:0/548347000 conn(0x7f77940b1d30 msgr2=0x7f77940b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 -- 172.21.15.64:0/548347000 shutdown_connections 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.011+0000 7f77998bf640 1 -- 172.21.15.64:0/548347000 wait complete. 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.012+0000 7f77998bf640 1 Processor -- start 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77998bf640 1 -- start start 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77998bf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cda20 0x7f779413d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77998bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77940cf610 0x7f77941f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77998bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7794145670 0x7f779415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77998bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7794145a50 con 0x7f77940cf610 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77998bf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779406e9f0 con 0x7f7794145670 2024-03-07T07:19:48.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77998bf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7794145bd0 con 0x7f77940cda20 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.013+0000 7f77927fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77940cf610 0x7f77941f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77940cf610 0x7f77941f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59366/0 (socket says 172.21.15.64:59366) 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 -- 172.21.15.64:0/211461085 learned_addr learned my addr 172.21.15.64:0/211461085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77937fe640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7794145670 0x7f779415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f7792ffd640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cda20 0x7f779413d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 -- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cda20 msgr2=0x7f779413d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cda20 0x7f779413d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 -- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7794145670 msgr2=0x7f779415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7794145670 0x7f779415b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 -- 172.21.15.64:0/211461085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77941f43e0 con 0x7f77940cf610 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77927fc640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77940cf610 0x7f77941f3ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f777c000c00 tx=0x7f777c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f778bfff640 1 -- 172.21.15.64:0/211461085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c032630 con 0x7f77940cf610 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f778bfff640 1 -- 172.21.15.64:0/211461085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c0327d0 con 0x7f77940cf610 2024-03-07T07:19:48.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779415bdc0 con 0x7f77940cf610 2024-03-07T07:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f778bfff640 1 -- 172.21.15.64:0/211461085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c032ba0 con 0x7f77940cf610 2024-03-07T07:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.014+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779413e0b0 con 0x7f77940cf610 2024-03-07T07:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.015+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7794062c90 con 0x7f77940cf610 2024-03-07T07:19:48.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.016+0000 7f778bfff640 1 -- 172.21.15.64:0/211461085 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f777c03b3f0 con 0x7f77940cf610 2024-03-07T07:19:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.016+0000 7f778bfff640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f777406b4d0 0x7f777406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.016+0000 7f778bfff640 1 -- 172.21.15.64:0/211461085 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f777c02f030 con 0x7f77940cf610 2024-03-07T07:19:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.016+0000 7f7792ffd640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f777406b4d0 0x7f777406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.017+0000 7f7792ffd640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f777406b4d0 0x7f777406d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7784005880 tx=0x7f7784009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.020+0000 7f778bfff640 1 -- 172.21.15.64:0/211461085 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f77941f43e0 con 0x7f77940cf610 2024-03-07T07:19:48.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.203+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f779413f0c0 con 0x7f77940cf610 2024-03-07T07:19:48.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.204+0000 7f778bfff640 1 -- 172.21.15.64:0/211461085 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f777c040070 con 0x7f77940cf610 2024-03-07T07:19:48.204 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:48.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f777406b4d0 msgr2=0x7f777406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f777406b4d0 0x7f777406d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7784005880 tx=0x7f7784009040 comp rx=0 tx=0).stop 2024-03-07T07:19:48.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77940cf610 msgr2=0x7f77941f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77940cf610 0x7f77941f3ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f777c000c00 tx=0x7f777c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 shutdown_connections 2024-03-07T07:19:48.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f777406b4d0 0x7f777406d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77940cf610 0x7f77941f3ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7794145670 0x7f779415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 --2- 172.21.15.64:0/211461085 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f77940cda20 0x7f779413d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 >> 172.21.15.64:0/211461085 conn(0x7f77940b1d30 msgr2=0x7f779415e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:48.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 shutdown_connections 2024-03-07T07:19:48.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.206+0000 7f77998bf640 1 -- 172.21.15.64:0/211461085 wait complete. 2024-03-07T07:19:48.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T07:19:48.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.320+0000 7f05a1372640 1 Processor -- start 2024-03-07T07:19:48.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.320+0000 7f05a1372640 1 -- start start 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f05a1372640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f059c070400 0x7f059c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f05a1372640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c070900 con 0x7f059c1440d0 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f05a1372640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c070be0 con 0x7f059c06dfc0 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f05a1372640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c071510 con 0x7f059c070400 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f059a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f059c070400 0x7f059c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f059a7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f059c070400 0x7f059c06da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59380/0 (socket says 172.21.15.64:59380) 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f059a7fc640 1 -- 172.21.15.64:0/764187105 learned_addr learned my addr 172.21.15.64:0/764187105 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f059a7fc640 1 -- 172.21.15.64:0/764187105 >> v1:172.21.15.120:6789/0 conn(0x7f059c06dfc0 legacy=0x7f059c0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f059a7fc640 1 -- 172.21.15.64:0/764187105 >> v1:172.21.15.64:6789/0 conn(0x7f059c1440d0 legacy=0x7f059c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.321+0000 7f059a7fc640 1 -- 172.21.15.64:0/764187105 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c070ec0 con 0x7f059c070400 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f059a7fc640 1 --2- 172.21.15.64:0/764187105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c06da80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f059000bb00 tx=0x7f059002df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ba55a4b9b3fac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f0599ffb640 1 -- 172.21.15.64:0/764187105 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590034070 con 0x7f059c070400 2024-03-07T07:19:48.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f0599ffb640 1 -- 172.21.15.64:0/764187105 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059002ecf0 con 0x7f059c070400 2024-03-07T07:19:48.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f0599ffb640 1 -- 172.21.15.64:0/764187105 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590038620 con 0x7f059c070400 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f05a1372640 1 -- 172.21.15.64:0/764187105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 msgr2=0x7f059c06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f05a1372640 1 --2- 172.21.15.64:0/764187105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c06da80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f059000bb00 tx=0x7f059002df40 comp rx=0 tx=0).stop 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f05a1372640 1 -- 172.21.15.64:0/764187105 shutdown_connections 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f05a1372640 1 --2- 172.21.15.64:0/764187105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c06da80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f05a1372640 1 -- 172.21.15.64:0/764187105 >> 172.21.15.64:0/764187105 conn(0x7f059c072490 msgr2=0x7f059c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.322+0000 7f05a1372640 1 -- 172.21.15.64:0/764187105 shutdown_connections 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.323+0000 7f05a1372640 1 -- 172.21.15.64:0/764187105 wait complete. 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 Processor -- start 2024-03-07T07:19:48.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 -- start start 2024-03-07T07:19:48.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f059c06dfc0 0x7f059c1f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f059c1440d0 0x7f059c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15ac10 con 0x7f059c070400 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c1444b0 con 0x7f059c1440d0 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.324+0000 7f05a1372640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15ad90 con 0x7f059c06dfc0 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59390/0 (socket says 172.21.15.64:59390) 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 -- 172.21.15.64:0/2388732343 learned_addr learned my addr 172.21.15.64:0/2388732343 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059b7fe640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f059c1440d0 0x7f059c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059affd640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f059c06dfc0 0x7f059c1f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 -- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f059c06dfc0 msgr2=0x7f059c1f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f059c06dfc0 0x7f059c1f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 -- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f059c1440d0 msgr2=0x7f059c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f059c1440d0 0x7f059c0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 -- 172.21.15.64:0/2388732343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c15b400 con 0x7f059c070400 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f059a7fc640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c15d7f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f059000bc20 tx=0x7f05900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f05837fe640 1 -- 172.21.15.64:0/2388732343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590047070 con 0x7f059c070400 2024-03-07T07:19:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f05837fe640 1 -- 172.21.15.64:0/2388732343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0590043870 con 0x7f059c070400 2024-03-07T07:19:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c15b6e0 con 0x7f059c070400 2024-03-07T07:19:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f05837fe640 1 -- 172.21.15.64:0/2388732343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590043c60 con 0x7f059c070400 2024-03-07T07:19:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.325+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c15bca0 con 0x7f059c070400 2024-03-07T07:19:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.326+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c071d70 con 0x7f059c070400 2024-03-07T07:19:48.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.327+0000 7f05837fe640 1 -- 172.21.15.64:0/2388732343 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f059004a070 con 0x7f059c070400 2024-03-07T07:19:48.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.327+0000 7f05837fe640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f058806b4d0 0x7f058806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.327+0000 7f05837fe640 1 -- 172.21.15.64:0/2388732343 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05900aead0 con 0x7f059c070400 2024-03-07T07:19:48.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.327+0000 7f059affd640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f058806b4d0 0x7f058806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.328+0000 7f059affd640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f058806b4d0 0x7f058806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f058c000f60 tx=0x7f058c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.332+0000 7f05837fe640 1 -- 172.21.15.64:0/2388732343 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f059c071d70 con 0x7f059c070400 2024-03-07T07:19:48.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.530+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f059c06e540 con 0x7f059c070400 2024-03-07T07:19:48.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.531+0000 7f05837fe640 1 -- 172.21.15.64:0/2388732343 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f059007d300 con 0x7f059c070400 2024-03-07T07:19:48.531 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-03-07T07:19:48.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f058806b4d0 msgr2=0x7f058806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f058806b4d0 0x7f058806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f058c000f60 tx=0x7f058c009040 comp rx=0 tx=0).stop 2024-03-07T07:19:48.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 msgr2=0x7f059c15d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c15d7f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f059000bc20 tx=0x7f05900046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 shutdown_connections 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f058806b4d0 0x7f058806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f059c070400 0x7f059c15d7f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f059c1440d0 0x7f059c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 --2- 172.21.15.64:0/2388732343 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f059c06dfc0 0x7f059c1f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 >> 172.21.15.64:0/2388732343 conn(0x7f059c072490 msgr2=0x7f059c15a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 shutdown_connections 2024-03-07T07:19:48.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.533+0000 7f05a1372640 1 -- 172.21.15.64:0/2388732343 wait complete. 2024-03-07T07:19:48.544 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-07T07:19:48.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:48.544 DEBUG:teuthology.orchestra.run.smithi064:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-07T07:19:48.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:19:48.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:48.572 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-07T07:19:48.639 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:48.639 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:48.655 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:48.655 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:48.709 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:48.710 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:48.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.866+0000 7f82f6310640 1 Processor -- start 2024-03-07T07:19:48.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.866+0000 7f82f6310640 1 -- start start 2024-03-07T07:19:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f6310640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f0144540 0x7f82f0162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f6310640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f82f0162ab0 0x7f82f0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f6310640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f00b4bd0 con 0x7f82f00b5dd0 2024-03-07T07:19:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f6310640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f0141700 con 0x7f82f0162ab0 2024-03-07T07:19:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f6310640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0161e00 con 0x7f82f0144540 2024-03-07T07:19:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82effff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f82f00b5dd0 0x7f82f00b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46094/0 (socket says 172.21.15.64:46094) 2024-03-07T07:19:48.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82effff640 1 -- 172.21.15.64:0/3593388032 learned_addr learned my addr 172.21.15.64:0/3593388032 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f48ad640 1 --2- 172.21.15.64:0/3593388032 >> v2:172.21.15.120:3300/0 conn(0x7f82f0162ab0 0x7f82f0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82ef7fe640 1 --2- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f0144540 0x7f82f0162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82eeffd640 1 -- 172.21.15.64:0/3593388032 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176844553 0 0) 0x7f82f00b4bd0 con 0x7f82f00b5dd0 2024-03-07T07:19:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82eeffd640 1 -- 172.21.15.64:0/3593388032 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d8003520 con 0x7f82f00b5dd0 2024-03-07T07:19:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f48ad640 1 -- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f0144540 msgr2=0x7f82f0162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f48ad640 1 --2- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f0144540 0x7f82f0162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f48ad640 1 -- 172.21.15.64:0/3593388032 >> v1:172.21.15.64:6789/0 conn(0x7f82f00b5dd0 legacy=0x7f82f00b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82f48ad640 1 -- 172.21.15.64:0/3593388032 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00b4eb0 con 0x7f82f0162ab0 2024-03-07T07:19:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.867+0000 7f82ef7fe640 1 --2- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f0144540 0x7f82f0162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.868+0000 7f82f48ad640 1 --2- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0162ab0 0x7f82f0071410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009d90 tx=0x7f82e802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6df24b82c4b0de3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.868+0000 7f82eeffd640 1 -- 172.21.15.64:0/3593388032 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8039070 con 0x7f82f0162ab0 2024-03-07T07:19:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.868+0000 7f82eeffd640 1 -- 172.21.15.64:0/3593388032 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e802cbc0 con 0x7f82f0162ab0 2024-03-07T07:19:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.868+0000 7f82eeffd640 1 -- 172.21.15.64:0/3593388032 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8034440 con 0x7f82f0162ab0 2024-03-07T07:19:48.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 -- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0162ab0 msgr2=0x7f82f0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 --2- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0162ab0 0x7f82f0071410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009d90 tx=0x7f82e802be10 comp rx=0 tx=0).stop 2024-03-07T07:19:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 -- 172.21.15.64:0/3593388032 shutdown_connections 2024-03-07T07:19:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 --2- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0162ab0 0x7f82f0071410 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 --2- 172.21.15.64:0/3593388032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f0144540 0x7f82f0162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 -- 172.21.15.64:0/3593388032 >> 172.21.15.64:0/3593388032 conn(0x7f82f00d0e70 msgr2=0x7f82f00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 -- 172.21.15.64:0/3593388032 shutdown_connections 2024-03-07T07:19:48.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.869+0000 7f82f6310640 1 -- 172.21.15.64:0/3593388032 wait complete. 2024-03-07T07:19:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.870+0000 7f82f6310640 1 Processor -- start 2024-03-07T07:19:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.870+0000 7f82f6310640 1 -- start start 2024-03-07T07:19:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82f6310640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82f00b5dd0 0x7f82f015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82f6310640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0144540 0x7f82f014d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82f6310640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f014dcc0 0x7f82f015f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82f6310640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00b61b0 con 0x7f82f00b5dd0 2024-03-07T07:19:48.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82f6310640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0162050 con 0x7f82f0144540 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82f6310640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0071950 con 0x7f82f014dcc0 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82f00b5dd0 0x7f82f015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82ef7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0144540 0x7f82f014d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82f00b5dd0 0x7f82f015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59400/0 (socket says 172.21.15.64:59400) 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82effff640 1 -- 172.21.15.64:0/3072875495 learned_addr learned my addr 172.21.15.64:0/3072875495 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82f48ad640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f014dcc0 0x7f82f015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82ef7fe640 1 -- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f014dcc0 msgr2=0x7f82f015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82ef7fe640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f014dcc0 0x7f82f015f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82ef7fe640 1 -- 172.21.15.64:0/3072875495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82f00b5dd0 msgr2=0x7f82f015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82ef7fe640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82f00b5dd0 0x7f82f015d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:48.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.871+0000 7f82ef7fe640 1 -- 172.21.15.64:0/3072875495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f015fd50 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.872+0000 7f82ef7fe640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0144540 0x7f82f014d780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e900 tx=0x7f82e000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.872+0000 7f82ecff9640 1 -- 172.21.15.64:0/3072875495 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0017070 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.872+0000 7f82ecff9640 1 -- 172.21.15.64:0/3072875495 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e000ca00 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.872+0000 7f82ecff9640 1 -- 172.21.15.64:0/3072875495 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000cdf0 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.872+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f0160030 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.872+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f01605f0 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.873+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f014e0d0 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.873+0000 7f82ecff9640 1 -- 172.21.15.64:0/3072875495 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f82e001a070 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.874+0000 7f82ecff9640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f82dc06b4d0 0x7f82dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.874+0000 7f82ecff9640 1 -- 172.21.15.64:0/3072875495 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82e008a390 con 0x7f82f0144540 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.874+0000 7f82effff640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f82dc06b4d0 0x7f82dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:48.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.874+0000 7f82effff640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f82dc06b4d0 0x7f82dc06d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005880 tx=0x7f82e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:48.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:48.878+0000 7f82ecff9640 1 -- 172.21.15.64:0/3072875495 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f82e0058b70 con 0x7f82f0144540 2024-03-07T07:19:49.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.060+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82f0062c90 con 0x7f82f0144540 2024-03-07T07:19:49.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.061+0000 7f82ecff9640 1 -- 172.21.15.64:0/3072875495 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f82e005cc10 con 0x7f82f0144540 2024-03-07T07:19:49.062 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:49.062 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f82dc06b4d0 msgr2=0x7f82dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f82dc06b4d0 0x7f82dc06d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005880 tx=0x7f82e4009040 comp rx=0 tx=0).stop 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0144540 msgr2=0x7f82f014d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0144540 0x7f82f014d780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f82e000e900 tx=0x7f82e000edc0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 shutdown_connections 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f82dc06b4d0 0x7f82dc06d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82f014dcc0 0x7f82f015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82f0144540 0x7f82f014d780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 --2- 172.21.15.64:0/3072875495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82f00b5dd0 0x7f82f015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 >> 172.21.15.64:0/3072875495 conn(0x7f82f00d0e70 msgr2=0x7f82f00d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 shutdown_connections 2024-03-07T07:19:49.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.063+0000 7f82f6310640 1 -- 172.21.15.64:0/3072875495 wait complete. 2024-03-07T07:19:49.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:49.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:49.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.178+0000 7f2d32d1d640 1 Processor -- start 2024-03-07T07:19:49.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.178+0000 7f2d32d1d640 1 -- start start 2024-03-07T07:19:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d32d1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c15e350 0x7f2d2c161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d32d1d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2d2c0b7200 0x7f2d2c1623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d32d1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d2c071280 0x7f2d2c071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d32d1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c0d2160 con 0x7f2d2c071280 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d32d1d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c071ff0 con 0x7f2d2c0b7200 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d32d1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0d23b0 con 0x7f2d2c15e350 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d312ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d2c071280 0x7f2d2c071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d312ba640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2d2c071280 0x7f2d2c071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59416/0 (socket says 172.21.15.64:59416) 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d30ab9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c15e350 0x7f2d2c161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d312ba640 1 -- 172.21.15.64:0/4049295217 learned_addr learned my addr 172.21.15.64:0/4049295217 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:49.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.179+0000 7f2d2bfff640 1 --2- 172.21.15.64:0/4049295217 >> v2:172.21.15.120:3300/0 conn(0x7f2d2c0b7200 0x7f2d2c1623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.180+0000 7f2d30ab9640 1 -- 172.21.15.64:0/4049295217 >> v2:172.21.15.120:3300/0 conn(0x7f2d2c0b7200 msgr2=0x7f2d2c1623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.180+0000 7f2d30ab9640 1 --2- 172.21.15.64:0/4049295217 >> v2:172.21.15.120:3300/0 conn(0x7f2d2c0b7200 0x7f2d2c1623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.180+0000 7f2d30ab9640 1 -- 172.21.15.64:0/4049295217 >> v2:172.21.15.64:3300/0 conn(0x7f2d2c071280 msgr2=0x7f2d2c071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.180+0000 7f2d30ab9640 1 --2- 172.21.15.64:0/4049295217 >> v2:172.21.15.64:3300/0 conn(0x7f2d2c071280 0x7f2d2c071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.180+0000 7f2d30ab9640 1 -- 172.21.15.64:0/4049295217 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c0d2690 con 0x7f2d2c15e350 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.180+0000 7f2d30ab9640 1 --2- 172.21.15.64:0/4049295217 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c15e350 0x7f2d2c161eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c068c40 tx=0x7f2d2402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37501011f5988bd server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.180+0000 7f2d2b7fe640 1 -- 172.21.15.64:0/4049295217 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24039070 con 0x7f2d2c15e350 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d2b7fe640 1 -- 172.21.15.64:0/4049295217 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2402cbd0 con 0x7f2d2c15e350 2024-03-07T07:19:49.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d2b7fe640 1 -- 172.21.15.64:0/4049295217 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24034440 con 0x7f2d2c15e350 2024-03-07T07:19:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 -- 172.21.15.64:0/4049295217 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c15e350 msgr2=0x7f2d2c161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/4049295217 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c15e350 0x7f2d2c161eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c068c40 tx=0x7f2d2402be20 comp rx=0 tx=0).stop 2024-03-07T07:19:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 -- 172.21.15.64:0/4049295217 shutdown_connections 2024-03-07T07:19:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/4049295217 >> v2:172.21.15.64:3300/0 conn(0x7f2d2c071280 0x7f2d2c071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/4049295217 >> v2:172.21.15.120:3300/0 conn(0x7f2d2c0b7200 0x7f2d2c1623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/4049295217 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c15e350 0x7f2d2c161eb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 -- 172.21.15.64:0/4049295217 >> 172.21.15.64:0/4049295217 conn(0x7f2d2c0d0e70 msgr2=0x7f2d2c0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 -- 172.21.15.64:0/4049295217 shutdown_connections 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.181+0000 7f2d32d1d640 1 -- 172.21.15.64:0/4049295217 wait complete. 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.182+0000 7f2d32d1d640 1 Processor -- start 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.182+0000 7f2d32d1d640 1 -- start start 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d32d1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c071280 0x7f2d2c1ee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d32d1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0b7200 0x7f2d2c1f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d32d1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2c15e350 0x7f2d2c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d32d1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c071c50 con 0x7f2d2c0b7200 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d32d1d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c15e730 con 0x7f2d2c15e350 2024-03-07T07:19:49.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d32d1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c071dd0 con 0x7f2d2c071280 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d2bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0b7200 0x7f2d2c1f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d2bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0b7200 0x7f2d2c1f42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59430/0 (socket says 172.21.15.64:59430) 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d2bfff640 1 -- 172.21.15.64:0/2226571442 learned_addr learned my addr 172.21.15.64:0/2226571442 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d312ba640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2c15e350 0x7f2d2c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.183+0000 7f2d30ab9640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c071280 0x7f2d2c1ee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d2bfff640 1 -- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c071280 msgr2=0x7f2d2c1ee1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d2bfff640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c071280 0x7f2d2c1ee1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d2bfff640 1 -- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2c15e350 msgr2=0x7f2d2c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d2bfff640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2c15e350 0x7f2d2c158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d2bfff640 1 -- 172.21.15.64:0/2226571442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c1ee6f0 con 0x7f2d2c0b7200 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d30ab9640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c071280 0x7f2d2c1ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:49.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d2bfff640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0b7200 0x7f2d2c1f42c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e8a0 tx=0x7f2d2000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d297fa640 1 -- 172.21.15.64:0/2226571442 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20017070 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d297fa640 1 -- 172.21.15.64:0/2226571442 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2000c940 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c1f4b10 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d297fa640 1 -- 172.21.15.64:0/2226571442 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000cd30 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.184+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c1f50d0 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.185+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c062c90 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.185+0000 7f2d297fa640 1 -- 172.21.15.64:0/2226571442 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2d2001a070 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.186+0000 7f2d297fa640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.186+0000 7f2d297fa640 1 -- 172.21.15.64:0/2226571442 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d20089fe0 con 0x7f2d2c0b7200 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.186+0000 7f2d30ab9640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.186+0000 7f2d30ab9640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2d1806b4d0 0x7f2d1806d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d24036000 tx=0x7f2d24030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.190+0000 7f2d297fa640 1 -- 172.21.15.64:0/2226571442 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d20058810 con 0x7f2d2c0b7200 2024-03-07T07:19:49.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.371+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d2c0b3a30 con 0x7f2d2c0b7200 2024-03-07T07:19:49.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.372+0000 7f2d297fa640 1 -- 172.21.15.64:0/2226571442 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d2005c8b0 con 0x7f2d2c0b7200 2024-03-07T07:19:49.372 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2d1806b4d0 msgr2=0x7f2d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2d1806b4d0 0x7f2d1806d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d24036000 tx=0x7f2d24030000 comp rx=0 tx=0).stop 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0b7200 msgr2=0x7f2d2c1f42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0b7200 0x7f2d2c1f42c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e8a0 tx=0x7f2d2000ed60 comp rx=0 tx=0).stop 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 shutdown_connections 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d2c0b7200 0x7f2d2c1f42c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d2c15e350 0x7f2d2c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 --2- 172.21.15.64:0/2226571442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2d2c071280 0x7f2d2c1ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 >> 172.21.15.64:0/2226571442 conn(0x7f2d2c0d0e70 msgr2=0x7f2d2c0740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 shutdown_connections 2024-03-07T07:19:49.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.374+0000 7f2d32d1d640 1 -- 172.21.15.64:0/2226571442 wait complete. 2024-03-07T07:19:49.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:49.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.488+0000 7f6ac7c5e640 1 Processor -- start 2024-03-07T07:19:49.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.489+0000 7f6ac7c5e640 1 -- start start 2024-03-07T07:19:49.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.489+0000 7f6ac7c5e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6ac015e350 0x7f6ac0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.489+0000 7f6ac7c5e640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac0071ff0 con 0x7f6ac00b7200 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.489+0000 7f6ac7c5e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac00d23b0 con 0x7f6ac0071990 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.489+0000 7f6ac7c5e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac00d2160 con 0x7f6ac015e350 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.489+0000 7f6ac51f9640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6ac00b7200 0x7f6ac0071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46096/0 (socket says 172.21.15.64:46096) 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.489+0000 7f6ac51f9640 1 -- 172.21.15.64:0/2563894183 learned_addr learned my addr 172.21.15.64:0/2563894183 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.490+0000 7f6ac59fa640 1 --2- 172.21.15.64:0/2563894183 >> v2:172.21.15.120:3300/0 conn(0x7f6ac015e350 0x7f6ac0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.490+0000 7f6ac49f8640 1 -- 172.21.15.64:0/2563894183 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155782198 0 0) 0x7f6ac0071ff0 con 0x7f6ac00b7200 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.490+0000 7f6ac49f8640 1 -- 172.21.15.64:0/2563894183 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aac003520 con 0x7f6ac00b7200 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.490+0000 7f6ac59fa640 1 -- 172.21.15.64:0/2563894183 >> v1:172.21.15.120:6789/0 conn(0x7f6ac0071990 legacy=0x7f6ac00d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.490+0000 7f6ac59fa640 1 -- 172.21.15.64:0/2563894183 >> v1:172.21.15.64:6789/0 conn(0x7f6ac00b7200 legacy=0x7f6ac0071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.490+0000 7f6ac59fa640 1 -- 172.21.15.64:0/2563894183 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac00d2690 con 0x7f6ac015e350 2024-03-07T07:19:49.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac59fa640 1 --2- 172.21.15.64:0/2563894183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac015e350 0x7f6ac0161eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0097a0 tx=0x7f6abc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1f42687e082110 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac49f8640 1 -- 172.21.15.64:0/2563894183 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc034070 con 0x7f6ac015e350 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac49f8640 1 -- 172.21.15.64:0/2563894183 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc030610 con 0x7f6ac015e350 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac49f8640 1 -- 172.21.15.64:0/2563894183 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0309e0 con 0x7f6ac015e350 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/2563894183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac015e350 msgr2=0x7f6ac0161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/2563894183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac015e350 0x7f6ac0161eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0097a0 tx=0x7f6abc02b960 comp rx=0 tx=0).stop 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/2563894183 shutdown_connections 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/2563894183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac015e350 0x7f6ac0161eb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.491+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/2563894183 >> 172.21.15.64:0/2563894183 conn(0x7f6ac00d0e70 msgr2=0x7f6ac00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.492+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/2563894183 shutdown_connections 2024-03-07T07:19:49.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.492+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/2563894183 wait complete. 2024-03-07T07:19:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 Processor -- start 2024-03-07T07:19:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 -- start start 2024-03-07T07:19:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac0071990 0x7f6ac01571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ac00b7200 0x7f6ac00d3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac015e350 0x7f6ac0154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01623f0 con 0x7f6ac015e350 2024-03-07T07:19:49.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac015e730 con 0x7f6ac0071990 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac7c5e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac0162570 con 0x7f6ac00b7200 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac61fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac015e350 0x7f6ac0154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac61fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac015e350 0x7f6ac0154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59440/0 (socket says 172.21.15.64:59440) 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac61fb640 1 -- 172.21.15.64:0/1521440288 learned_addr learned my addr 172.21.15.64:0/1521440288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac59fa640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac0071990 0x7f6ac01571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.493+0000 7f6ac51f9640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ac00b7200 0x7f6ac00d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac61fb640 1 -- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ac00b7200 msgr2=0x7f6ac00d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac61fb640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ac00b7200 0x7f6ac00d3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac61fb640 1 -- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac0071990 msgr2=0x7f6ac01571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac61fb640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac0071990 0x7f6ac01571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac61fb640 1 -- 172.21.15.64:0/1521440288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac015f830 con 0x7f6ac015e350 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac61fb640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac015e350 0x7f6ac0154490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800c7d0 tx=0x7f6ab800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ab67fc640 1 -- 172.21.15.64:0/1521440288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8003b70 con 0x7f6ac015e350 2024-03-07T07:19:49.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ab67fc640 1 -- 172.21.15.64:0/1521440288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab8003d30 con 0x7f6ac015e350 2024-03-07T07:19:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac015fb10 con 0x7f6ac015e350 2024-03-07T07:19:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ab67fc640 1 -- 172.21.15.64:0/1521440288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab800aa90 con 0x7f6ac015e350 2024-03-07T07:19:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.494+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac0160580 con 0x7f6ac015e350 2024-03-07T07:19:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.495+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac00b51a0 con 0x7f6ac015e350 2024-03-07T07:19:49.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.495+0000 7f6ab67fc640 1 -- 172.21.15.64:0/1521440288 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6ab800ac30 con 0x7f6ac015e350 2024-03-07T07:19:49.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.496+0000 7f6ab67fc640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6aa406b4d0 0x7f6aa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.496+0000 7f6ab67fc640 1 -- 172.21.15.64:0/1521440288 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ab808a010 con 0x7f6ac015e350 2024-03-07T07:19:49.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.496+0000 7f6ac59fa640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6aa406b4d0 0x7f6aa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.496+0000 7f6ac59fa640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6aa406b4d0 0x7f6aa406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6abc036000 tx=0x7f6abc03b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.500+0000 7f6ab67fc640 1 -- 172.21.15.64:0/1521440288 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ac00b51a0 con 0x7f6ac015e350 2024-03-07T07:19:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.678+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ac0160780 con 0x7f6ac015e350 2024-03-07T07:19:49.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.678+0000 7f6ab67fc640 1 -- 172.21.15.64:0/1521440288 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ac0160780 con 0x7f6ac015e350 2024-03-07T07:19:49.679 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6aa406b4d0 msgr2=0x7f6aa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6aa406b4d0 0x7f6aa406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6abc036000 tx=0x7f6abc03b040 comp rx=0 tx=0).stop 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac015e350 msgr2=0x7f6ac0154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac015e350 0x7f6ac0154490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800c7d0 tx=0x7f6ab800cc90 comp rx=0 tx=0).stop 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 shutdown_connections 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6aa406b4d0 0x7f6aa406d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ac015e350 0x7f6ac0154490 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ac00b7200 0x7f6ac00d3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 --2- 172.21.15.64:0/1521440288 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ac0071990 0x7f6ac01571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 >> 172.21.15.64:0/1521440288 conn(0x7f6ac00d0e70 msgr2=0x7f6ac00d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 shutdown_connections 2024-03-07T07:19:49.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.680+0000 7f6ac7c5e640 1 -- 172.21.15.64:0/1521440288 wait complete. 2024-03-07T07:19:49.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:49.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.795+0000 7ff64fc2b640 1 Processor -- start 2024-03-07T07:19:49.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.795+0000 7ff64fc2b640 1 -- start start 2024-03-07T07:19:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64fc2b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff64815e350 0x7ff648161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64fc2b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6480b7200 0x7ff6481623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64fc2b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff648071ff0 con 0x7ff648071280 2024-03-07T07:19:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64fc2b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480d2160 con 0x7ff64815e350 2024-03-07T07:19:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64fc2b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480d23b0 con 0x7ff6480b7200 2024-03-07T07:19:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64d9c7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff64815e350 0x7ff648161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64e1c8640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff648071280 0x7ff648071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46106/0 (socket says 172.21.15.64:46106) 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64e1c8640 1 -- 172.21.15.64:0/3805710940 learned_addr learned my addr 172.21.15.64:0/3805710940 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.796+0000 7ff64d1c6640 1 --2- 172.21.15.64:0/3805710940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6480b7200 0x7ff6481623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64d9c7640 1 -- 172.21.15.64:0/3805710940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6480b7200 msgr2=0x7ff6481623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64d9c7640 1 --2- 172.21.15.64:0/3805710940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6480b7200 0x7ff6481623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64d9c7640 1 -- 172.21.15.64:0/3805710940 >> v1:172.21.15.64:6789/0 conn(0x7ff648071280 legacy=0x7ff648071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64d9c7640 1 -- 172.21.15.64:0/3805710940 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480d2690 con 0x7ff64815e350 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64c9c5640 1 -- 172.21.15.64:0/3805710940 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810898739 0 0) 0x7ff648071ff0 con 0x7ff648071280 2024-03-07T07:19:49.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64d1c6640 1 --2- 172.21.15.64:0/3805710940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6480b7200 0x7ff6481623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64d9c7640 1 --2- 172.21.15.64:0/3805710940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64815e350 0x7ff648161eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff638009d90 tx=0x7ff63802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e0f6582718a1b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64c9c5640 1 -- 172.21.15.64:0/3805710940 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638039070 con 0x7ff64815e350 2024-03-07T07:19:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.797+0000 7ff64c9c5640 1 -- 172.21.15.64:0/3805710940 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63802cc50 con 0x7ff64815e350 2024-03-07T07:19:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64c9c5640 1 -- 172.21.15.64:0/3805710940 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638034440 con 0x7ff64815e350 2024-03-07T07:19:49.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3805710940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64815e350 msgr2=0x7ff648161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3805710940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64815e350 0x7ff648161eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff638009d90 tx=0x7ff63802bea0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3805710940 shutdown_connections 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3805710940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6480b7200 0x7ff6481623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3805710940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64815e350 0x7ff648161eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3805710940 >> 172.21.15.64:0/3805710940 conn(0x7ff6480d0e70 msgr2=0x7ff6480d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3805710940 shutdown_connections 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.798+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3805710940 wait complete. 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.799+0000 7ff64fc2b640 1 Processor -- start 2024-03-07T07:19:49.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.799+0000 7ff64fc2b640 1 -- start start 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64fc2b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff648071280 0x7ff6480c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64fc2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6480b7200 0x7ff648160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64fc2b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff64815e350 0x7ff648161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64fc2b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480d2e90 con 0x7ff6480b7200 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64fc2b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64815e730 con 0x7ff64815e350 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64fc2b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480d3010 con 0x7ff648071280 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6480b7200 0x7ff648160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6480b7200 0x7ff648160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59454/0 (socket says 172.21.15.64:59454) 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 -- 172.21.15.64:0/3673540327 learned_addr learned my addr 172.21.15.64:0/3673540327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d9c7640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff648071280 0x7ff6480c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64e1c8640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff64815e350 0x7ff648161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 -- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff648071280 msgr2=0x7ff6480c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff648071280 0x7ff6480c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 -- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff64815e350 msgr2=0x7ff648161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff64815e350 0x7ff648161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.800+0000 7ff64d1c6640 1 -- 172.21.15.64:0/3673540327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480ca010 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.801+0000 7ff64d1c6640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6480b7200 0x7ff648160b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00ebe0 tx=0x7ff63c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.801+0000 7ff6467fc640 1 -- 172.21.15.64:0/3673540327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c017070 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.801+0000 7ff6467fc640 1 -- 172.21.15.64:0/3673540327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c00cc30 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.801+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6481f4a40 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.801+0000 7ff6467fc640 1 -- 172.21.15.64:0/3673540327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c013470 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.801+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6480c7fb0 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.802+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648062c90 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.802+0000 7ff6467fc640 1 -- 172.21.15.64:0/3673540327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff63c01a070 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.802+0000 7ff6467fc640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.803+0000 7ff6467fc640 1 -- 172.21.15.64:0/3673540327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff63c08a020 con 0x7ff6480b7200 2024-03-07T07:19:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.803+0000 7ff64d9c7640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.803+0000 7ff64d9c7640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff63406b4d0 0x7ff63406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff638036000 tx=0x7ff638030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.807+0000 7ff6467fc640 1 -- 172.21.15.64:0/3673540327 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff63c0588e0 con 0x7ff6480b7200 2024-03-07T07:19:49.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.985+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6480cf170 con 0x7ff6480b7200 2024-03-07T07:19:49.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.986+0000 7ff6467fc640 1 -- 172.21.15.64:0/3673540327 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff63c05c980 con 0x7ff6480b7200 2024-03-07T07:19:49.986 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:49.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff63406b4d0 msgr2=0x7ff63406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff63406b4d0 0x7ff63406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff638036000 tx=0x7ff638030000 comp rx=0 tx=0).stop 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6480b7200 msgr2=0x7ff648160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6480b7200 0x7ff648160b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00ebe0 tx=0x7ff63c00c520 comp rx=0 tx=0).stop 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 shutdown_connections 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6480b7200 0x7ff648160b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff64815e350 0x7ff648161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 --2- 172.21.15.64:0/3673540327 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff648071280 0x7ff6480c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 >> 172.21.15.64:0/3673540327 conn(0x7ff6480d0e70 msgr2=0x7ff6480b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 shutdown_connections 2024-03-07T07:19:49.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:49.987+0000 7ff64fc2b640 1 -- 172.21.15.64:0/3673540327 wait complete. 2024-03-07T07:19:49.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T07:19:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.100+0000 7fcdb1f54640 1 Processor -- start 2024-03-07T07:19:50.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.100+0000 7fcdb1f54640 1 -- start start 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdb1f54640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac147790 0x7fcdac0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdb1f54640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0b4520 con 0x7fcdac0b55c0 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdb1f54640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0b4800 con 0x7fcdac15e0d0 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdb1f54640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0b5f60 con 0x7fcdac147790 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdaaffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fcdac0b55c0 0x7fcdac147080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:46110/0 (socket says 172.21.15.64:46110) 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdaaffd640 1 -- 172.21.15.64:0/1842764947 learned_addr learned my addr 172.21.15.64:0/1842764947 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdabfff640 1 --2- 172.21.15.64:0/1842764947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac147790 0x7fcdac0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdaa7fc640 1 -- 172.21.15.64:0/1842764947 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414695196 0 0) 0x7fcdac0b4520 con 0x7fcdac0b55c0 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdaa7fc640 1 -- 172.21.15.64:0/1842764947 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd98003520 con 0x7fcdac0b55c0 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdaa7fc640 1 -- 172.21.15.64:0/1842764947 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193305678 0 0) 0x7fcdac0b4800 con 0x7fcdac15e0d0 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdaa7fc640 1 -- 172.21.15.64:0/1842764947 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac0b4520 con 0x7fcdac15e0d0 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdabfff640 1 -- 172.21.15.64:0/1842764947 >> v1:172.21.15.120:6789/0 conn(0x7fcdac15e0d0 legacy=0x7fcdac1417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.101+0000 7fcdabfff640 1 -- 172.21.15.64:0/1842764947 >> v1:172.21.15.64:6789/0 conn(0x7fcdac0b55c0 legacy=0x7fcdac147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.102+0000 7fcdabfff640 1 -- 172.21.15.64:0/1842764947 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0b4ae0 con 0x7fcdac147790 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.102+0000 7fcdabfff640 1 --2- 172.21.15.64:0/1842764947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac147790 0x7fcdac0b5a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009dc0 tx=0x7fcda402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0e61f58c499398 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.102+0000 7fcdaa7fc640 1 -- 172.21.15.64:0/1842764947 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4039070 con 0x7fcdac147790 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.102+0000 7fcdaa7fc640 1 -- 172.21.15.64:0/1842764947 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda402cbb0 con 0x7fcdac147790 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdaa7fc640 1 -- 172.21.15.64:0/1842764947 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4034440 con 0x7fcdac147790 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdb1f54640 1 -- 172.21.15.64:0/1842764947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac147790 msgr2=0x7fcdac0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/1842764947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac147790 0x7fcdac0b5a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009dc0 tx=0x7fcda402be00 comp rx=0 tx=0).stop 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdb1f54640 1 -- 172.21.15.64:0/1842764947 shutdown_connections 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/1842764947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac147790 0x7fcdac0b5a20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdb1f54640 1 -- 172.21.15.64:0/1842764947 >> 172.21.15.64:0/1842764947 conn(0x7fcdac071300 msgr2=0x7fcdac071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdb1f54640 1 -- 172.21.15.64:0/1842764947 shutdown_connections 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.103+0000 7fcdb1f54640 1 -- 172.21.15.64:0/1842764947 wait complete. 2024-03-07T07:19:50.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.104+0000 7fcdb1f54640 1 Processor -- start 2024-03-07T07:19:50.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.104+0000 7fcdb1f54640 1 -- start start 2024-03-07T07:19:50.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdb1f54640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdac0b55c0 0x7fcdac140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdb1f54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdac147790 0x7fcdac1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdb1f54640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac15e0d0 0x7fcdac1435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdb1f54640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac141ee0 con 0x7fcdac147790 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdb1f54640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0b50a0 con 0x7fcdac0b55c0 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdb1f54640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac15e4b0 con 0x7fcdac15e0d0 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdac147790 0x7fcdac1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdac147790 0x7fcdac1430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59464/0 (socket says 172.21.15.64:59464) 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 -- 172.21.15.64:0/2616517863 learned_addr learned my addr 172.21.15.64:0/2616517863 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdabfff640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac15e0d0 0x7fcdac1435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdab7fe640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdac0b55c0 0x7fcdac140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 -- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac15e0d0 msgr2=0x7fcdac1435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac15e0d0 0x7fcdac1435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 -- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdac0b55c0 msgr2=0x7fcdac140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdac0b55c0 0x7fcdac140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.105+0000 7fcdaaffd640 1 -- 172.21.15.64:0/2616517863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac1f4e00 con 0x7fcdac147790 2024-03-07T07:19:50.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.106+0000 7fcdaaffd640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdac147790 0x7fcdac1430b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00c4d0 tx=0x7fcd9c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.106+0000 7fcda3fff640 1 -- 172.21.15.64:0/2616517863 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c017070 con 0x7fcdac147790 2024-03-07T07:19:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.106+0000 7fcda3fff640 1 -- 172.21.15.64:0/2616517863 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c005d00 con 0x7fcdac147790 2024-03-07T07:19:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.106+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac1f4ae0 con 0x7fcdac147790 2024-03-07T07:19:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.106+0000 7fcda3fff640 1 -- 172.21.15.64:0/2616517863 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0072b0 con 0x7fcdac147790 2024-03-07T07:19:50.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.106+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac1f53a0 con 0x7fcdac147790 2024-03-07T07:19:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.107+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac062c90 con 0x7fcdac147790 2024-03-07T07:19:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.107+0000 7fcda3fff640 1 -- 172.21.15.64:0/2616517863 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcd9c01a070 con 0x7fcdac147790 2024-03-07T07:19:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.108+0000 7fcda3fff640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.108+0000 7fcda3fff640 1 -- 172.21.15.64:0/2616517863 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd9c089530 con 0x7fcdac147790 2024-03-07T07:19:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.108+0000 7fcdab7fe640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.108+0000 7fcdab7fe640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd94007fd0 tx=0x7fcd94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:50.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.112+0000 7fcda3fff640 1 -- 172.21.15.64:0/2616517863 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcd9c057e20 con 0x7fcdac147790 2024-03-07T07:19:50.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.305+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcdac141bd0 con 0x7fcdac147790 2024-03-07T07:19:50.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.305+0000 7fcda3fff640 1 -- 172.21.15.64:0/2616517863 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd9c05bec0 con 0x7fcdac147790 2024-03-07T07:19:50.305 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-03-07T07:19:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fcd8c06b4d0 msgr2=0x7fcd8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd94007fd0 tx=0x7fcd94009040 comp rx=0 tx=0).stop 2024-03-07T07:19:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdac147790 msgr2=0x7fcdac1430b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdac147790 0x7fcdac1430b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00c4d0 tx=0x7fcd9c00c990 comp rx=0 tx=0).stop 2024-03-07T07:19:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 shutdown_connections 2024-03-07T07:19:50.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdac147790 0x7fcdac1430b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcdac15e0d0 0x7fcdac1435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 --2- 172.21.15.64:0/2616517863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcdac0b55c0 0x7fcdac140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 >> 172.21.15.64:0/2616517863 conn(0x7fcdac071300 msgr2=0x7fcdac144c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 shutdown_connections 2024-03-07T07:19:50.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.307+0000 7fcdb1f54640 1 -- 172.21.15.64:0/2616517863 wait complete. 2024-03-07T07:19:50.318 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-07T07:19:50.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:50.318 DEBUG:teuthology.orchestra.run.smithi120:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-07T07:19:50.349 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:19:50.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:50.350 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-07T07:19:50.496 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:50.496 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:50.511 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:50.512 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:50.568 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:19:50.568 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:50.623 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.765+0000 7f8671634640 1 Processor -- start 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.765+0000 7f8671634640 1 -- start start 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f8671634640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f866c0aa030 0x7f866c0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f8671634640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f866c15ce20 0x7f866c0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f8671634640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0aa410 con 0x7f866c06e800 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f8671634640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c06da80 con 0x7f866c15ce20 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f8671634640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c15d440 con 0x7f866c0aa030 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f866c15ce20 0x7f866c0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866b7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f866c15ce20 0x7f866c0b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46978/0 (socket says 172.21.15.64:46978) 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866b7fe640 1 -- 172.21.15.64:0/3965154228 learned_addr learned my addr 172.21.15.64:0/3965154228 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866affd640 1 --2- 172.21.15.64:0/3965154228 >> v2:172.21.15.120:3300/0 conn(0x7f866c0aa030 0x7f866c0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866b7fe640 1 -- 172.21.15.64:0/3965154228 >> v2:172.21.15.120:3300/0 conn(0x7f866c0aa030 msgr2=0x7f866c0b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866b7fe640 1 --2- 172.21.15.64:0/3965154228 >> v2:172.21.15.120:3300/0 conn(0x7f866c0aa030 0x7f866c0b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866b7fe640 1 -- 172.21.15.64:0/3965154228 >> v1:172.21.15.120:6789/0 conn(0x7f866c06e800 legacy=0x7f866c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.766+0000 7f866b7fe640 1 -- 172.21.15.64:0/3965154228 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c06dd20 con 0x7f866c15ce20 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.767+0000 7f866b7fe640 1 --2- 172.21.15.64:0/3965154228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c15ce20 0x7f866c0b2c20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f866c067440 tx=0x7f865c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad5819d033267d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.767+0000 7f8669ffb640 1 -- 172.21.15.64:0/3965154228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c02cb20 con 0x7f866c15ce20 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.767+0000 7f8669ffb640 1 -- 172.21.15.64:0/3965154228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c02ccc0 con 0x7f866c15ce20 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.767+0000 7f8669ffb640 1 -- 172.21.15.64:0/3965154228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c034610 con 0x7f866c15ce20 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 -- 172.21.15.64:0/3965154228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c15ce20 msgr2=0x7f866c0b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 --2- 172.21.15.64:0/3965154228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c15ce20 0x7f866c0b2c20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f866c067440 tx=0x7f865c02bf80 comp rx=0 tx=0).stop 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 -- 172.21.15.64:0/3965154228 shutdown_connections 2024-03-07T07:19:50.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 --2- 172.21.15.64:0/3965154228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c15ce20 0x7f866c0b2c20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 --2- 172.21.15.64:0/3965154228 >> v2:172.21.15.120:3300/0 conn(0x7f866c0aa030 0x7f866c0b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 -- 172.21.15.64:0/3965154228 >> 172.21.15.64:0/3965154228 conn(0x7f866c072490 msgr2=0x7f866c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 -- 172.21.15.64:0/3965154228 shutdown_connections 2024-03-07T07:19:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.768+0000 7f8671634640 1 -- 172.21.15.64:0/3965154228 wait complete. 2024-03-07T07:19:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.769+0000 7f8671634640 1 Processor -- start 2024-03-07T07:19:50.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.769+0000 7f8671634640 1 -- start start 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f8671634640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c06e800 0x7f866c155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f8671634640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f866c0aa030 0x7f866c1f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f8671634640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f866c15ce20 0x7f866c139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f8671634640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c070400 con 0x7f866c06e800 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f8671634640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c06e2e0 con 0x7f866c15ce20 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f8671634640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c070580 con 0x7f866c0aa030 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c06e800 0x7f866c155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c06e800 0x7f866c155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46994/0 (socket says 172.21.15.64:46994) 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 -- 172.21.15.64:0/4104384752 learned_addr learned my addr 172.21.15.64:0/4104384752 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866b7fe640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f866c15ce20 0x7f866c139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866a7fc640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f866c0aa030 0x7f866c1f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 -- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f866c0aa030 msgr2=0x7f866c1f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f866c0aa030 0x7f866c1f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 -- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f866c15ce20 msgr2=0x7f866c139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f866c15ce20 0x7f866c139c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.770+0000 7f866affd640 1 -- 172.21.15.64:0/4104384752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c138bf0 con 0x7f866c06e800 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.771+0000 7f866affd640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c06e800 0x7f866c155e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f866400e9b0 tx=0x7f866400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.771+0000 7f86637fe640 1 -- 172.21.15.64:0/4104384752 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664017070 con 0x7f866c06e800 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.771+0000 7f86637fe640 1 -- 172.21.15.64:0/4104384752 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866400c9a0 con 0x7f866c06e800 2024-03-07T07:19:50.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.771+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c138ed0 con 0x7f866c06e800 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.771+0000 7f86637fe640 1 -- 172.21.15.64:0/4104384752 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400cd90 con 0x7f866c06e800 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.771+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c139490 con 0x7f866c06e800 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.772+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c062c90 con 0x7f866c06e800 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.772+0000 7f86637fe640 1 -- 172.21.15.64:0/4104384752 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f866401a070 con 0x7f866c06e800 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.773+0000 7f86637fe640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f864c06b4d0 0x7f864c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.773+0000 7f86637fe640 1 -- 172.21.15.64:0/4104384752 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f866408a4a0 con 0x7f866c06e800 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.773+0000 7f866a7fc640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f864c06b4d0 0x7f864c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:50.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.773+0000 7f866a7fc640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f864c06b4d0 0x7f864c06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f866c0cf9f0 tx=0x7f8654009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:50.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.777+0000 7f86637fe640 1 -- 172.21.15.64:0/4104384752 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8664058d60 con 0x7f866c06e800 2024-03-07T07:19:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.958+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f866c1420d0 con 0x7f866c06e800 2024-03-07T07:19:50.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.959+0000 7f86637fe640 1 -- 172.21.15.64:0/4104384752 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f866405ce00 con 0x7f866c06e800 2024-03-07T07:19:50.960 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:50.960 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:50.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f864c06b4d0 msgr2=0x7f864c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f864c06b4d0 0x7f864c06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f866c0cf9f0 tx=0x7f8654009040 comp rx=0 tx=0).stop 2024-03-07T07:19:50.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c06e800 msgr2=0x7f866c155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:50.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c06e800 0x7f866c155e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f866400e9b0 tx=0x7f866400ee70 comp rx=0 tx=0).stop 2024-03-07T07:19:50.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 shutdown_connections 2024-03-07T07:19:50.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f864c06b4d0 0x7f864c06d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f866c15ce20 0x7f866c139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f866c0aa030 0x7f866c1f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 --2- 172.21.15.64:0/4104384752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f866c06e800 0x7f866c155e80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:50.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 >> 172.21.15.64:0/4104384752 conn(0x7f866c072490 msgr2=0x7f866c141030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:50.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 shutdown_connections 2024-03-07T07:19:50.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:50.961+0000 7f8671634640 1 -- 172.21.15.64:0/4104384752 wait complete. 2024-03-07T07:19:50.972 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:50.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:51.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.077+0000 7f3165c0d640 1 Processor -- start 2024-03-07T07:19:51.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.077+0000 7f3165c0d640 1 -- start start 2024-03-07T07:19:51.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f3165c0d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f316015e870 0x7f31600cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f3165c0d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3160070eb0 con 0x7f31600d3310 2024-03-07T07:19:51.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f3165c0d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3160071190 con 0x7f31600cea70 2024-03-07T07:19:51.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f3165c0d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3160071470 con 0x7f316015e870 2024-03-07T07:19:51.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f31600d3310 0x7f3160167e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45318/0 (socket says 172.21.15.64:45318) 2024-03-07T07:19:51.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315ffff640 1 -- 172.21.15.64:0/1872795445 learned_addr learned my addr 172.21.15.64:0/1872795445 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315f7fe640 1 --2- 172.21.15.64:0/1872795445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f316015e870 0x7f31600cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315e7fc640 1 -- 172.21.15.64:0/1872795445 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243702395 0 0) 0x7f3160070eb0 con 0x7f31600d3310 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315e7fc640 1 -- 172.21.15.64:0/1872795445 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3148003520 con 0x7f31600d3310 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315e7fc640 1 -- 172.21.15.64:0/1872795445 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940636712 0 0) 0x7f3160071190 con 0x7f31600cea70 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315e7fc640 1 -- 172.21.15.64:0/1872795445 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3160070eb0 con 0x7f31600cea70 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315f7fe640 1 -- 172.21.15.64:0/1872795445 >> v1:172.21.15.120:6789/0 conn(0x7f31600cea70 legacy=0x7f316015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.078+0000 7f315f7fe640 1 -- 172.21.15.64:0/1872795445 >> v1:172.21.15.64:6789/0 conn(0x7f31600d3310 legacy=0x7f3160167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.079+0000 7f315f7fe640 1 -- 172.21.15.64:0/1872795445 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160071750 con 0x7f316015e870 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.079+0000 7f315f7fe640 1 --2- 172.21.15.64:0/1872795445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f316015e870 0x7f31600cf190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31500097d0 tx=0x7f315002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e44254c9ef14f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.079+0000 7f315e7fc640 1 -- 172.21.15.64:0/1872795445 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150035020 con 0x7f316015e870 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.079+0000 7f315e7fc640 1 -- 172.21.15.64:0/1872795445 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3150031890 con 0x7f316015e870 2024-03-07T07:19:51.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.079+0000 7f315e7fc640 1 -- 172.21.15.64:0/1872795445 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150031c40 con 0x7f316015e870 2024-03-07T07:19:51.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.080+0000 7f3165c0d640 1 -- 172.21.15.64:0/1872795445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f316015e870 msgr2=0x7f31600cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.080+0000 7f3165c0d640 1 --2- 172.21.15.64:0/1872795445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f316015e870 0x7f31600cf190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f31500097d0 tx=0x7f315002b930 comp rx=0 tx=0).stop 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.080+0000 7f3165c0d640 1 -- 172.21.15.64:0/1872795445 shutdown_connections 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.080+0000 7f3165c0d640 1 --2- 172.21.15.64:0/1872795445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f316015e870 0x7f31600cf190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.080+0000 7f3165c0d640 1 -- 172.21.15.64:0/1872795445 >> 172.21.15.64:0/1872795445 conn(0x7f31600d11e0 msgr2=0x7f31600d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.080+0000 7f3165c0d640 1 -- 172.21.15.64:0/1872795445 shutdown_connections 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.080+0000 7f3165c0d640 1 -- 172.21.15.64:0/1872795445 wait complete. 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.081+0000 7f3165c0d640 1 Processor -- start 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.081+0000 7f3165c0d640 1 -- start start 2024-03-07T07:19:51.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f3165c0d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31600cea70 0x7f3160152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f3165c0d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31600d3310 0x7f3160155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f3165c0d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f316015e870 0x7f316015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f3165c0d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600cf6d0 con 0x7f316015e870 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f3165c0d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3160070b60 con 0x7f31600d3310 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f3165c0d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31600cf850 con 0x7f31600cea70 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f316015e870 0x7f316015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f316015e870 0x7f316015a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46998/0 (socket says 172.21.15.64:46998) 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 -- 172.21.15.64:0/147236662 learned_addr learned my addr 172.21.15.64:0/147236662 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315f7fe640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31600cea70 0x7f3160152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315effd640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31600d3310 0x7f3160155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 -- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31600cea70 msgr2=0x7f3160152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31600cea70 0x7f3160152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 -- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31600d3310 msgr2=0x7f3160155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31600d3310 0x7f3160155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315ffff640 1 -- 172.21.15.64:0/147236662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160154bb0 con 0x7f316015e870 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.082+0000 7f315f7fe640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31600cea70 0x7f3160152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.083+0000 7f315ffff640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f316015e870 0x7f316015a070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f315800cb20 tx=0x7f315800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.083+0000 7f3147fff640 1 -- 172.21.15.64:0/147236662 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158017070 con 0x7f316015e870 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.083+0000 7f3147fff640 1 -- 172.21.15.64:0/147236662 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315800aca0 con 0x7f316015e870 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.083+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3160154e90 con 0x7f316015e870 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.083+0000 7f3147fff640 1 -- 172.21.15.64:0/147236662 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158012600 con 0x7f316015e870 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.083+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3160155450 con 0x7f316015e870 2024-03-07T07:19:51.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.084+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3160062c90 con 0x7f316015e870 2024-03-07T07:19:51.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.084+0000 7f3147fff640 1 -- 172.21.15.64:0/147236662 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f315801a070 con 0x7f316015e870 2024-03-07T07:19:51.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.085+0000 7f3147fff640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f314c06b500 0x7f314c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.085+0000 7f3147fff640 1 -- 172.21.15.64:0/147236662 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3158089eb0 con 0x7f316015e870 2024-03-07T07:19:51.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.085+0000 7f315f7fe640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f314c06b500 0x7f314c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.085+0000 7f315f7fe640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f314c06b500 0x7f314c06d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3150036000 tx=0x7f3150008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.089+0000 7f3147fff640 1 -- 172.21.15.64:0/147236662 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f31580586e0 con 0x7f316015e870 2024-03-07T07:19:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.270+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31600b7580 con 0x7f316015e870 2024-03-07T07:19:51.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.270+0000 7f3147fff640 1 -- 172.21.15.64:0/147236662 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f315805c780 con 0x7f316015e870 2024-03-07T07:19:51.270 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f314c06b500 msgr2=0x7f314c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f314c06b500 0x7f314c06d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3150036000 tx=0x7f3150008030 comp rx=0 tx=0).stop 2024-03-07T07:19:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f316015e870 msgr2=0x7f316015a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f316015e870 0x7f316015a070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f315800cb20 tx=0x7f315800a060 comp rx=0 tx=0).stop 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 shutdown_connections 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f314c06b500 0x7f314c06d9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f316015e870 0x7f316015a070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31600d3310 0x7f3160155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 --2- 172.21.15.64:0/147236662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31600cea70 0x7f3160152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 >> 172.21.15.64:0/147236662 conn(0x7f31600d11e0 msgr2=0x7f3160161350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 shutdown_connections 2024-03-07T07:19:51.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.272+0000 7f3165c0d640 1 -- 172.21.15.64:0/147236662 wait complete. 2024-03-07T07:19:51.283 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.387+0000 7f4a636a1640 1 Processor -- start 2024-03-07T07:19:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.387+0000 7f4a636a1640 1 -- start start 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a636a1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a5c162110 0x7f4a5c1624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a636a1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4a5c070bc0 0x7f4a5c15e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a636a1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c162a30 con 0x7f4a5c071050 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a636a1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5c071610 con 0x7f4a5c162110 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a636a1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5c0718f0 con 0x7f4a5c070bc0 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a6143d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a5c162110 0x7f4a5c1624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a6143d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4a5c162110 0x7f4a5c1624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47004/0 (socket says 172.21.15.64:47004) 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a6143d640 1 -- 172.21.15.64:0/27141379 learned_addr learned my addr 172.21.15.64:0/27141379 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a60c3c640 1 --2- 172.21.15.64:0/27141379 >> v2:172.21.15.120:3300/0 conn(0x7f4a5c070bc0 0x7f4a5c15e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a60c3c640 1 -- 172.21.15.64:0/27141379 >> v2:172.21.15.64:3300/0 conn(0x7f4a5c162110 msgr2=0x7f4a5c1624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a60c3c640 1 --2- 172.21.15.64:0/27141379 >> v2:172.21.15.64:3300/0 conn(0x7f4a5c162110 0x7f4a5c1624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.388+0000 7f4a60c3c640 1 -- 172.21.15.64:0/27141379 >> v1:172.21.15.120:6789/0 conn(0x7f4a5c071050 legacy=0x7f4a5c15ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.389+0000 7f4a60c3c640 1 -- 172.21.15.64:0/27141379 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c071bd0 con 0x7f4a5c070bc0 2024-03-07T07:19:51.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.389+0000 7f4a5bfff640 1 -- 172.21.15.64:0/27141379 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642859255 0 0) 0x7f4a5c162a30 con 0x7f4a5c071050 2024-03-07T07:19:51.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.389+0000 7f4a6143d640 1 --2- 172.21.15.64:0/27141379 >> v2:172.21.15.64:3300/0 conn(0x7f4a5c162110 0x7f4a5c1624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.389+0000 7f4a60c3c640 1 --2- 172.21.15.64:0/27141379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 0x7f4a5c15e8d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c08d010 tx=0x7f4a5002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9381b8ff052c598f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.389+0000 7f4a5bfff640 1 -- 172.21.15.64:0/27141379 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500315a0 con 0x7f4a5c070bc0 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.389+0000 7f4a5bfff640 1 -- 172.21.15.64:0/27141379 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a50037070 con 0x7f4a5c070bc0 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.389+0000 7f4a5bfff640 1 -- 172.21.15.64:0/27141379 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50031960 con 0x7f4a5c070bc0 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 -- 172.21.15.64:0/27141379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 msgr2=0x7f4a5c15e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 --2- 172.21.15.64:0/27141379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 0x7f4a5c15e8d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c08d010 tx=0x7f4a5002b840 comp rx=0 tx=0).stop 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 -- 172.21.15.64:0/27141379 shutdown_connections 2024-03-07T07:19:51.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 --2- 172.21.15.64:0/27141379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 0x7f4a5c15e8d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 --2- 172.21.15.64:0/27141379 >> v2:172.21.15.64:3300/0 conn(0x7f4a5c162110 0x7f4a5c1624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 -- 172.21.15.64:0/27141379 >> 172.21.15.64:0/27141379 conn(0x7f4a5c0d0e10 msgr2=0x7f4a5c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:51.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 -- 172.21.15.64:0/27141379 shutdown_connections 2024-03-07T07:19:51.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.390+0000 7f4a636a1640 1 -- 172.21.15.64:0/27141379 wait complete. 2024-03-07T07:19:51.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.391+0000 7f4a636a1640 1 Processor -- start 2024-03-07T07:19:51.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.391+0000 7f4a636a1640 1 -- start start 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a636a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 0x7f4a5c14a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a636a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a5c071050 0x7f4a5c0b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a636a1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4a5c162110 0x7f4a5c1470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a636a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c162e00 con 0x7f4a5c071050 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a636a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c0d21d0 con 0x7f4a5c070bc0 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a636a1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c162f80 con 0x7f4a5c162110 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a5c071050 0x7f4a5c0b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a5c071050 0x7f4a5c0b8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47006/0 (socket says 172.21.15.64:47006) 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 -- 172.21.15.64:0/1291160205 learned_addr learned my addr 172.21.15.64:0/1291160205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a6143d640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 0x7f4a5c14a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a61c3e640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4a5c162110 0x7f4a5c1470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 -- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4a5c162110 msgr2=0x7f4a5c1470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4a5c162110 0x7f4a5c1470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 -- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 msgr2=0x7f4a5c14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 0x7f4a5c14a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.392+0000 7f4a60c3c640 1 -- 172.21.15.64:0/1291160205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c0b3930 con 0x7f4a5c071050 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.393+0000 7f4a60c3c640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a5c071050 0x7f4a5c0b8960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009890 tx=0x7f4a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.393+0000 7f4a59ffb640 1 -- 172.21.15.64:0/1291160205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50004230 con 0x7f4a5c071050 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.393+0000 7f4a59ffb640 1 -- 172.21.15.64:0/1291160205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a500043d0 con 0x7f4a5c071050 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.393+0000 7f4a59ffb640 1 -- 172.21.15.64:0/1291160205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50003850 con 0x7f4a5c071050 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.393+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c0b3bd0 con 0x7f4a5c071050 2024-03-07T07:19:51.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.393+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c0b8160 con 0x7f4a5c071050 2024-03-07T07:19:51.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.394+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c062c90 con 0x7f4a5c071050 2024-03-07T07:19:51.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.394+0000 7f4a59ffb640 1 -- 172.21.15.64:0/1291160205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4a500039f0 con 0x7f4a5c071050 2024-03-07T07:19:51.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.395+0000 7f4a59ffb640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4a4806b4d0 0x7f4a4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.395+0000 7f4a59ffb640 1 -- 172.21.15.64:0/1291160205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a5002f030 con 0x7f4a5c071050 2024-03-07T07:19:51.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.395+0000 7f4a6143d640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4a4806b4d0 0x7f4a4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.395+0000 7f4a6143d640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4a4806b4d0 0x7f4a4806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a54006ca0 tx=0x7f4a54009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.399+0000 7f4a59ffb640 1 -- 172.21.15.64:0/1291160205 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4a50041070 con 0x7f4a5c071050 2024-03-07T07:19:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.579+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a5c0cab80 con 0x7f4a5c071050 2024-03-07T07:19:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.579+0000 7f4a59ffb640 1 -- 172.21.15.64:0/1291160205 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a5007aad0 con 0x7f4a5c071050 2024-03-07T07:19:51.579 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4a4806b4d0 msgr2=0x7f4a4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4a4806b4d0 0x7f4a4806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a54006ca0 tx=0x7f4a54009240 comp rx=0 tx=0).stop 2024-03-07T07:19:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a5c071050 msgr2=0x7f4a5c0b8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a5c071050 0x7f4a5c0b8960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009890 tx=0x7f4a500046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 shutdown_connections 2024-03-07T07:19:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4a4806b4d0 0x7f4a4806d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4a5c071050 0x7f4a5c0b8960 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4a5c162110 0x7f4a5c1470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 --2- 172.21.15.64:0/1291160205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4a5c070bc0 0x7f4a5c14a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 >> 172.21.15.64:0/1291160205 conn(0x7f4a5c0d0e10 msgr2=0x7f4a5c15c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 shutdown_connections 2024-03-07T07:19:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.581+0000 7f4a636a1640 1 -- 172.21.15.64:0/1291160205 wait complete. 2024-03-07T07:19:51.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:51.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.696+0000 7f0b11310640 1 Processor -- start 2024-03-07T07:19:51.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.696+0000 7f0b11310640 1 -- start start 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.696+0000 7f0b11310640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b0c15f800 0x7f0b0c09a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.696+0000 7f0b11310640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0b0c15fbe0 0x7f0b0c09af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.696+0000 7f0b11310640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c0d1090 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.696+0000 7f0b11310640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0d1370 con 0x7f0b0c15f800 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.696+0000 7f0b11310640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0d1650 con 0x7f0b0c15fbe0 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0b0c15f800 0x7f0b0c09a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f0b0c0b0fb0 0x7f0b0c0cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45328/0 (socket says 172.21.15.64:45328) 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0b7fe640 1 -- 172.21.15.64:0/1704379472 learned_addr learned my addr 172.21.15.64:0/1704379472 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b03fff640 1 --2- 172.21.15.64:0/1704379472 >> v2:172.21.15.120:3300/0 conn(0x7f0b0c15fbe0 0x7f0b0c09af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0affd640 1 -- 172.21.15.64:0/1704379472 >> v2:172.21.15.120:3300/0 conn(0x7f0b0c15fbe0 msgr2=0x7f0b0c09af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0affd640 1 --2- 172.21.15.64:0/1704379472 >> v2:172.21.15.120:3300/0 conn(0x7f0b0c15fbe0 0x7f0b0c09af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0affd640 1 -- 172.21.15.64:0/1704379472 >> v1:172.21.15.64:6789/0 conn(0x7f0b0c0b0fb0 legacy=0x7f0b0c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0a7fc640 1 -- 172.21.15.64:0/1704379472 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254596111 0 0) 0x7f0b0c0d1090 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.697+0000 7f0b0affd640 1 -- 172.21.15.64:0/1704379472 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c0d1930 con 0x7f0b0c15f800 2024-03-07T07:19:51.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b0affd640 1 --2- 172.21.15.64:0/1704379472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c15f800 0x7f0b0c09a9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0097a0 tx=0x7f0afc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3941bb15326ee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b0a7fc640 1 -- 172.21.15.64:0/1704379472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc035020 con 0x7f0b0c15f800 2024-03-07T07:19:51.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b0a7fc640 1 -- 172.21.15.64:0/1704379472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc031830 con 0x7f0b0c15f800 2024-03-07T07:19:51.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b0a7fc640 1 -- 172.21.15.64:0/1704379472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc031c00 con 0x7f0b0c15f800 2024-03-07T07:19:51.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b11310640 1 -- 172.21.15.64:0/1704379472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c15f800 msgr2=0x7f0b0c09a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b11310640 1 --2- 172.21.15.64:0/1704379472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c15f800 0x7f0b0c09a9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0097a0 tx=0x7f0afc02b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b11310640 1 -- 172.21.15.64:0/1704379472 shutdown_connections 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b11310640 1 --2- 172.21.15.64:0/1704379472 >> v2:172.21.15.120:3300/0 conn(0x7f0b0c15fbe0 0x7f0b0c09af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b11310640 1 --2- 172.21.15.64:0/1704379472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c15f800 0x7f0b0c09a9d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.698+0000 7f0b11310640 1 -- 172.21.15.64:0/1704379472 >> 172.21.15.64:0/1704379472 conn(0x7f0b0c0b3b90 msgr2=0x7f0b0c0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.699+0000 7f0b11310640 1 -- 172.21.15.64:0/1704379472 shutdown_connections 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.699+0000 7f0b11310640 1 -- 172.21.15.64:0/1704379472 wait complete. 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 Processor -- start 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 -- start start 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c0b0fb0 0x7f0b0c1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b0c15f800 0x7f0b0c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c15fbe0 0x7f0b0c0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0cdc60 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0b15e0 con 0x7f0b0c15fbe0 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b11310640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0cdde0 con 0x7f0b0c15f800 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c0b0fb0 0x7f0b0c1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.700+0000 7f0b0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c0b0fb0 0x7f0b0c1588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47026/0 (socket says 172.21.15.64:47026) 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0affd640 1 -- 172.21.15.64:0/1608638736 learned_addr learned my addr 172.21.15.64:0/1608638736 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0b7fe640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c15fbe0 0x7f0b0c0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b03fff640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b0c15f800 0x7f0b0c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0affd640 1 -- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b0c15f800 msgr2=0x7f0b0c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0affd640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b0c15f800 0x7f0b0c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0affd640 1 -- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c15fbe0 msgr2=0x7f0b0c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0affd640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c15fbe0 0x7f0b0c0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0affd640 1 -- 172.21.15.64:0/1608638736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c0b31b0 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b0affd640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c0b0fb0 0x7f0b0c1588d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0afc02bee0 tx=0x7f0afc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b037fe640 1 -- 172.21.15.64:0/1608638736 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc041070 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b037fe640 1 -- 172.21.15.64:0/1608638736 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc03f070 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b037fe640 1 -- 172.21.15.64:0/1608638736 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc035040 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c071ae0 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.701+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c0720a0 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.702+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c0722a0 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.703+0000 7f0b037fe640 1 -- 172.21.15.64:0/1608638736 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0afc038020 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.703+0000 7f0b037fe640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0aec06b4d0 0x7f0aec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:51.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.703+0000 7f0b037fe640 1 -- 172.21.15.64:0/1608638736 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0afc0aba30 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.703+0000 7f0b03fff640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0aec06b4d0 0x7f0aec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:51.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.704+0000 7f0b03fff640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0aec06b4d0 0x7f0aec06d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0b6380 tx=0x7f0af4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:51.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.708+0000 7f0b037fe640 1 -- 172.21.15.64:0/1608638736 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0b0c0722a0 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.887+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b0c062c90 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.887+0000 7f0b037fe640 1 -- 172.21.15.64:0/1608638736 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0afc031800 con 0x7f0b0c0b0fb0 2024-03-07T07:19:51.888 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:51.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0aec06b4d0 msgr2=0x7f0aec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0aec06b4d0 0x7f0aec06d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0b6380 tx=0x7f0af4009190 comp rx=0 tx=0).stop 2024-03-07T07:19:51.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c0b0fb0 msgr2=0x7f0b0c1588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:51.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c0b0fb0 0x7f0b0c1588d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0afc02bee0 tx=0x7f0afc0046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 shutdown_connections 2024-03-07T07:19:51.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0aec06b4d0 0x7f0aec06d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c15fbe0 0x7f0b0c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b0c15f800 0x7f0b0c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 --2- 172.21.15.64:0/1608638736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c0b0fb0 0x7f0b0c1588d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:51.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 >> 172.21.15.64:0/1608638736 conn(0x7f0b0c0b3b90 msgr2=0x7f0b0c0cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:51.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 shutdown_connections 2024-03-07T07:19:51.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:51.889+0000 7f0b11310640 1 -- 172.21.15.64:0/1608638736 wait complete. 2024-03-07T07:19:51.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T07:19:52.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.003+0000 7f6ed3c87640 1 Processor -- start 2024-03-07T07:19:52.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.003+0000 7f6ed3c87640 1 -- start start 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed3c87640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ecc15fbb0 0x7f6ecc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed3c87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc15c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed3c87640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ecc0731b0 con 0x7f6ecc072950 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed3c87640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc072f60 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed3c87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc073490 con 0x7f6ecc0cdee0 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed1a23640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f6ecc15fbb0 0x7f6ecc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed1222640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f6ecc072950 0x7f6ecc15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45334/0 (socket says 172.21.15.64:45334) 2024-03-07T07:19:52.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed1222640 1 -- 172.21.15.64:0/4101774631 learned_addr learned my addr 172.21.15.64:0/4101774631 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed2224640 1 --2- 172.21.15.64:0/4101774631 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed2224640 1 -- 172.21.15.64:0/4101774631 >> v2:172.21.15.64:3300/0 conn(0x7f6ecc15fbb0 msgr2=0x7f6ecc0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed2224640 1 --2- 172.21.15.64:0/4101774631 >> v2:172.21.15.64:3300/0 conn(0x7f6ecc15fbb0 0x7f6ecc0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed2224640 1 -- 172.21.15.64:0/4101774631 >> v1:172.21.15.64:6789/0 conn(0x7f6ecc072950 legacy=0x7f6ecc15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed0a21640 1 -- 172.21.15.64:0/4101774631 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513698315 0 0) 0x7f6ecc0731b0 con 0x7f6ecc072950 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed2224640 1 -- 172.21.15.64:0/4101774631 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc072150 con 0x7f6ecc0cdee0 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.004+0000 7f6ed1a23640 1 --2- 172.21.15.64:0/4101774631 >> v2:172.21.15.64:3300/0 conn(0x7f6ecc15fbb0 0x7f6ecc0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.005+0000 7f6ed2224640 1 --2- 172.21.15.64:0/4101774631 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc15c550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009d90 tx=0x7f6ec402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143692bdc4dec05 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.005+0000 7f6ed0a21640 1 -- 172.21.15.64:0/4101774631 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4039070 con 0x7f6ecc0cdee0 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.005+0000 7f6ed0a21640 1 -- 172.21.15.64:0/4101774631 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec402cbc0 con 0x7f6ecc0cdee0 2024-03-07T07:19:52.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.005+0000 7f6ed0a21640 1 -- 172.21.15.64:0/4101774631 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4034440 con 0x7f6ecc0cdee0 2024-03-07T07:19:52.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 -- 172.21.15.64:0/4101774631 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 msgr2=0x7f6ecc15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/4101774631 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc15c550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009d90 tx=0x7f6ec402be10 comp rx=0 tx=0).stop 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 -- 172.21.15.64:0/4101774631 shutdown_connections 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/4101774631 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc15c550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/4101774631 >> v2:172.21.15.64:3300/0 conn(0x7f6ecc15fbb0 0x7f6ecc0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 -- 172.21.15.64:0/4101774631 >> 172.21.15.64:0/4101774631 conn(0x7f6ecc0b3120 msgr2=0x7f6ecc0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 -- 172.21.15.64:0/4101774631 shutdown_connections 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.006+0000 7f6ed3c87640 1 -- 172.21.15.64:0/4101774631 wait complete. 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.007+0000 7f6ed3c87640 1 Processor -- start 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.007+0000 7f6ed3c87640 1 -- start start 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.007+0000 7f6ed3c87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc14eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.007+0000 7f6ed3c87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ecc15fbb0 0x7f6ecc0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed3c87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ecc0c5da0 0x7f6ecc15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed3c87640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0ce2c0 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed3c87640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc160980 con 0x7f6ecc0c5da0 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed3c87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0b1a00 con 0x7f6ecc0cdee0 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ecc15fbb0 0x7f6ecc0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ecc15fbb0 0x7f6ecc0c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47054/0 (socket says 172.21.15.64:47054) 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 -- 172.21.15.64:0/1300832299 learned_addr learned my addr 172.21.15.64:0/1300832299 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1a23640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc14eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed2224640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ecc0c5da0 0x7f6ecc15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 -- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 msgr2=0x7f6ecc14eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc14eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 -- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ecc0c5da0 msgr2=0x7f6ecc15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ecc0c5da0 0x7f6ecc15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 -- 172.21.15.64:0/1300832299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc1f2870 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed1222640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ecc15fbb0 0x7f6ecc0c5860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c410 tx=0x7f6ebc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ec27fc640 1 -- 172.21.15.64:0/1300832299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0074e0 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ec27fc640 1 -- 172.21.15.64:0/1300832299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc0047e0 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc1f2b10 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ec27fc640 1 -- 172.21.15.64:0/1300832299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc004bd0 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.008+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc1f30d0 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.009+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc062c90 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.010+0000 7f6ec27fc640 1 -- 172.21.15.64:0/1300832299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6ebc011070 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.010+0000 7f6ec27fc640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6eb006b500 0x7f6eb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.010+0000 7f6ec27fc640 1 -- 172.21.15.64:0/1300832299 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ebc089bd0 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.010+0000 7f6ed1a23640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6eb006b500 0x7f6eb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.011+0000 7f6ed1a23640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6eb006b500 0x7f6eb006d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0cc4a0 tx=0x7f6ec8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.014+0000 7f6ec27fc640 1 -- 172.21.15.64:0/1300832299 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ebc058410 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.207+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ecc068340 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.207+0000 7f6ec27fc640 1 -- 172.21.15.64:0/1300832299 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6ebc05c4b0 con 0x7f6ecc15fbb0 2024-03-07T07:19:52.207 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6eb006b500 msgr2=0x7f6eb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6eb006b500 0x7f6eb006d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0cc4a0 tx=0x7f6ec8009190 comp rx=0 tx=0).stop 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ecc15fbb0 msgr2=0x7f6ecc0c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ecc15fbb0 0x7f6ecc0c5860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c410 tx=0x7f6ebc00c8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 shutdown_connections 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f6eb006b500 0x7f6eb006d9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ecc15fbb0 0x7f6ecc0c5860 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ecc0c5da0 0x7f6ecc15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 --2- 172.21.15.64:0/1300832299 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ecc0cdee0 0x7f6ecc14eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 >> 172.21.15.64:0/1300832299 conn(0x7f6ecc0b3120 msgr2=0x7f6ecc0740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:52.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 shutdown_connections 2024-03-07T07:19:52.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.209+0000 7f6ed3c87640 1 -- 172.21.15.64:0/1300832299 wait complete. 2024-03-07T07:19:52.219 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-07T07:19:52.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:52.219 DEBUG:teuthology.orchestra.run.smithi064:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-07T07:19:52.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:19:52.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:52.247 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-07T07:19:52.301 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:52.301 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:52.317 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:52.317 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:52.372 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:19:52.372 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:19:52.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:52.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.531+0000 7f17dd114640 1 Processor -- start 2024-03-07T07:19:52.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.531+0000 7f17dd114640 1 -- start start 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17dd114640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f17d80af950 0x7f17d80b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17dd114640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d806f140 con 0x7f17d80ce5f0 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17dd114640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d806f420 con 0x7f17d815b1b0 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17dd114640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d8071610 con 0x7f17d80af950 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f17d80ce5f0 0x7f17d8071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45350/0 (socket says 172.21.15.64:45350) 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d659c640 1 -- 172.21.15.64:0/3040805912 learned_addr learned my addr 172.21.15.64:0/3040805912 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d759e640 1 --2- 172.21.15.64:0/3040805912 >> v2:172.21.15.120:3300/0 conn(0x7f17d80af950 0x7f17d80b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d5d9b640 1 -- 172.21.15.64:0/3040805912 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380660364 0 0) 0x7f17d806f140 con 0x7f17d80ce5f0 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d5d9b640 1 -- 172.21.15.64:0/3040805912 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c0003520 con 0x7f17d80ce5f0 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d5d9b640 1 -- 172.21.15.64:0/3040805912 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12877740 0 0) 0x7f17d806f420 con 0x7f17d815b1b0 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d5d9b640 1 -- 172.21.15.64:0/3040805912 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17d806f140 con 0x7f17d815b1b0 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d759e640 1 -- 172.21.15.64:0/3040805912 >> v1:172.21.15.120:6789/0 conn(0x7f17d815b1b0 legacy=0x7f17d80cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d759e640 1 -- 172.21.15.64:0/3040805912 >> v1:172.21.15.64:6789/0 conn(0x7f17d80ce5f0 legacy=0x7f17d8071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.532+0000 7f17d759e640 1 -- 172.21.15.64:0/3040805912 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d8071860 con 0x7f17d80af950 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.533+0000 7f17d759e640 1 --2- 172.21.15.64:0/3040805912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d80af950 0x7f17d80b3c10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009d90 tx=0x7f17d002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf7ab95c6687d79 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.533+0000 7f17d5d9b640 1 -- 172.21.15.64:0/3040805912 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0039070 con 0x7f17d80af950 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.533+0000 7f17d5d9b640 1 -- 172.21.15.64:0/3040805912 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d002cc60 con 0x7f17d80af950 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.533+0000 7f17d5d9b640 1 -- 172.21.15.64:0/3040805912 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0034660 con 0x7f17d80af950 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.534+0000 7f17dd114640 1 -- 172.21.15.64:0/3040805912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d80af950 msgr2=0x7f17d80b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.534+0000 7f17dd114640 1 --2- 172.21.15.64:0/3040805912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d80af950 0x7f17d80b3c10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009d90 tx=0x7f17d002beb0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.534+0000 7f17dd114640 1 -- 172.21.15.64:0/3040805912 shutdown_connections 2024-03-07T07:19:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.534+0000 7f17dd114640 1 --2- 172.21.15.64:0/3040805912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d80af950 0x7f17d80b3c10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.534+0000 7f17dd114640 1 -- 172.21.15.64:0/3040805912 >> 172.21.15.64:0/3040805912 conn(0x7f17d80b3120 msgr2=0x7f17d80b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.534+0000 7f17dd114640 1 -- 172.21.15.64:0/3040805912 shutdown_connections 2024-03-07T07:19:52.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.534+0000 7f17dd114640 1 -- 172.21.15.64:0/3040805912 wait complete. 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.535+0000 7f17dd114640 1 Processor -- start 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.535+0000 7f17dd114640 1 -- start start 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17dd114640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d80af950 0x7f17d80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17dd114640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17d80ce5f0 0x7f17d8143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17dd114640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d815b1b0 0x7f17d815d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17dd114640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d815b590 con 0x7f17d80ce5f0 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17dd114640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d806ee30 con 0x7f17d815b1b0 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17dd114640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d815b710 con 0x7f17d80af950 2024-03-07T07:19:52.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17d80ce5f0 0x7f17d8143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17d80ce5f0 0x7f17d8143cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47068/0 (socket says 172.21.15.64:47068) 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 -- 172.21.15.64:0/3296907149 learned_addr learned my addr 172.21.15.64:0/3296907149 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d759e640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d815b1b0 0x7f17d815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d6d9d640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d80af950 0x7f17d80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 -- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d80af950 msgr2=0x7f17d80b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d80af950 0x7f17d80b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 -- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d815b1b0 msgr2=0x7f17d815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d815b1b0 0x7f17d815d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 -- 172.21.15.64:0/3296907149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d8144580 con 0x7f17d80ce5f0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.536+0000 7f17d659c640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17d80ce5f0 0x7f17d8143cf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00c500 tx=0x7f17cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.537+0000 7f17bf7fe640 1 -- 172.21.15.64:0/3296907149 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc017070 con 0x7f17d80ce5f0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.537+0000 7f17bf7fe640 1 -- 172.21.15.64:0/3296907149 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc005ce0 con 0x7f17d80ce5f0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.537+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d8144860 con 0x7f17d80ce5f0 2024-03-07T07:19:52.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.537+0000 7f17bf7fe640 1 -- 172.21.15.64:0/3296907149 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0072f0 con 0x7f17d80ce5f0 2024-03-07T07:19:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.537+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d80b4850 con 0x7f17d80ce5f0 2024-03-07T07:19:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.538+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d8062c90 con 0x7f17d80ce5f0 2024-03-07T07:19:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.538+0000 7f17bf7fe640 1 -- 172.21.15.64:0/3296907149 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f17cc01a070 con 0x7f17d80ce5f0 2024-03-07T07:19:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.538+0000 7f17bf7fe640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17c406b4d0 0x7f17c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.538+0000 7f17bf7fe640 1 -- 172.21.15.64:0/3296907149 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17cc0894e0 con 0x7f17d80ce5f0 2024-03-07T07:19:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.538+0000 7f17d6d9d640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17c406b4d0 0x7f17c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.539+0000 7f17d6d9d640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17c406b4d0 0x7f17c406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f17c8007fd0 tx=0x7f17c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.543+0000 7f17bf7fe640 1 -- 172.21.15.64:0/3296907149 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f17cc057d20 con 0x7f17d80ce5f0 2024-03-07T07:19:52.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.726+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17d80cc660 con 0x7f17d80ce5f0 2024-03-07T07:19:52.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.727+0000 7f17bf7fe640 1 -- 172.21.15.64:0/3296907149 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f17cc05bdc0 con 0x7f17d80ce5f0 2024-03-07T07:19:52.727 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:52.727 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17c406b4d0 msgr2=0x7f17c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17c406b4d0 0x7f17c406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f17c8007fd0 tx=0x7f17c8009040 comp rx=0 tx=0).stop 2024-03-07T07:19:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17d80ce5f0 msgr2=0x7f17d8143cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17d80ce5f0 0x7f17d8143cf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00c500 tx=0x7f17cc00c9c0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 shutdown_connections 2024-03-07T07:19:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17c406b4d0 0x7f17c406d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17d80ce5f0 0x7f17d8143cf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d815b1b0 0x7f17d815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 --2- 172.21.15.64:0/3296907149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d80af950 0x7f17d80b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 >> 172.21.15.64:0/3296907149 conn(0x7f17d80b3120 msgr2=0x7f17d81597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 shutdown_connections 2024-03-07T07:19:52.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.729+0000 7f17dd114640 1 -- 172.21.15.64:0/3296907149 wait complete. 2024-03-07T07:19:52.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:52.740 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.844+0000 7f68b54a1640 1 Processor -- start 2024-03-07T07:19:52.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.844+0000 7f68b54a1640 1 -- start start 2024-03-07T07:19:52.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68b54a1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68b015e350 0x7f68b0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68b54a1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f68b00b7200 0x7f68b01623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68b54a1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b0071ff0 con 0x7f68b0071280 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68b54a1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b00d2160 con 0x7f68b015e350 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68b54a1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b00d23b0 con 0x7f68b00b7200 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68aeffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68b015e350 0x7f68b0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68aeffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f68b015e350 0x7f68b0161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47074/0 (socket says 172.21.15.64:47074) 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68aeffd640 1 -- 172.21.15.64:0/256847992 learned_addr learned my addr 172.21.15.64:0/256847992 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.845+0000 7f68ae7fc640 1 --2- 172.21.15.64:0/256847992 >> v2:172.21.15.120:3300/0 conn(0x7f68b00b7200 0x7f68b01623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.846+0000 7f68aeffd640 1 -- 172.21.15.64:0/256847992 >> v2:172.21.15.120:3300/0 conn(0x7f68b00b7200 msgr2=0x7f68b01623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.846+0000 7f68aeffd640 1 --2- 172.21.15.64:0/256847992 >> v2:172.21.15.120:3300/0 conn(0x7f68b00b7200 0x7f68b01623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.846+0000 7f68aeffd640 1 -- 172.21.15.64:0/256847992 >> v1:172.21.15.120:6789/0 conn(0x7f68b0071280 legacy=0x7f68b00716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.846+0000 7f68aeffd640 1 -- 172.21.15.64:0/256847992 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00d2690 con 0x7f68b015e350 2024-03-07T07:19:52.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.846+0000 7f68aeffd640 1 --2- 172.21.15.64:0/256847992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b015e350 0x7f68b0161eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b9b00 tx=0x7f68a802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdb50dcafc8c5a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68adffb640 1 -- 172.21.15.64:0/256847992 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8034070 con 0x7f68b015e350 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68adffb640 1 -- 172.21.15.64:0/256847992 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8030860 con 0x7f68b015e350 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68adffb640 1 -- 172.21.15.64:0/256847992 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8030c30 con 0x7f68b015e350 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 -- 172.21.15.64:0/256847992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b015e350 msgr2=0x7f68b0161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 --2- 172.21.15.64:0/256847992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b015e350 0x7f68b0161eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b9b00 tx=0x7f68a802b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 -- 172.21.15.64:0/256847992 shutdown_connections 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 --2- 172.21.15.64:0/256847992 >> v2:172.21.15.120:3300/0 conn(0x7f68b00b7200 0x7f68b01623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 --2- 172.21.15.64:0/256847992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b015e350 0x7f68b0161eb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 -- 172.21.15.64:0/256847992 >> 172.21.15.64:0/256847992 conn(0x7f68b00d0e70 msgr2=0x7f68b00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 -- 172.21.15.64:0/256847992 shutdown_connections 2024-03-07T07:19:52.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.847+0000 7f68b54a1640 1 -- 172.21.15.64:0/256847992 wait complete. 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 Processor -- start 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 -- start start 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0071280 0x7f68b00ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b00b7200 0x7f68b01f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68b015e350 0x7f68b015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00d2e90 con 0x7f68b0071280 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b015e730 con 0x7f68b00b7200 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68b54a1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00d3010 con 0x7f68b015e350 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0071280 0x7f68b00ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0071280 0x7f68b00ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47088/0 (socket says 172.21.15.64:47088) 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68aeffd640 1 -- 172.21.15.64:0/1283167433 learned_addr learned my addr 172.21.15.64:0/1283167433 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68ae7fc640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b00b7200 0x7f68b01f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.849+0000 7f68af7fe640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68b015e350 0x7f68b015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68aeffd640 1 -- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68b015e350 msgr2=0x7f68b015adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68aeffd640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68b015e350 0x7f68b015adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68aeffd640 1 -- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b00b7200 msgr2=0x7f68b01f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68aeffd640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b00b7200 0x7f68b01f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68aeffd640 1 -- 172.21.15.64:0/1283167433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b01f8720 con 0x7f68b0071280 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68aeffd640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0071280 0x7f68b00ca530 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f68a802b9e0 tx=0x7f68a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68a77fe640 1 -- 172.21.15.64:0/1283167433 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8034040 con 0x7f68b0071280 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68a77fe640 1 -- 172.21.15.64:0/1283167433 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8039d90 con 0x7f68b0071280 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b015e9e0 con 0x7f68b0071280 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68a77fe640 1 -- 172.21.15.64:0/1283167433 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8043610 con 0x7f68b0071280 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.850+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b015ec80 con 0x7f68b0071280 2024-03-07T07:19:52.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.851+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b0062c90 con 0x7f68b0071280 2024-03-07T07:19:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.852+0000 7f68a77fe640 1 -- 172.21.15.64:0/1283167433 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f68a804a060 con 0x7f68b0071280 2024-03-07T07:19:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.852+0000 7f68a77fe640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.852+0000 7f68a77fe640 1 -- 172.21.15.64:0/1283167433 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68a80ac050 con 0x7f68b0071280 2024-03-07T07:19:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.852+0000 7f68ae7fc640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:52.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.852+0000 7f68ae7fc640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f689006b4d0 0x7f689006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f68b01602f0 tx=0x7f6898009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:52.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:52.856+0000 7f68a77fe640 1 -- 172.21.15.64:0/1283167433 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68b01f8720 con 0x7f68b0071280 2024-03-07T07:19:53.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.037+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68b00717d0 con 0x7f68b0071280 2024-03-07T07:19:53.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.038+0000 7f68a77fe640 1 -- 172.21.15.64:0/1283167433 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68a803e070 con 0x7f68b0071280 2024-03-07T07:19:53.038 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f689006b4d0 msgr2=0x7f689006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f689006b4d0 0x7f689006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f68b01602f0 tx=0x7f6898009190 comp rx=0 tx=0).stop 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0071280 msgr2=0x7f68b00ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0071280 0x7f68b00ca530 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f68a802b9e0 tx=0x7f68a80046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 shutdown_connections 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68b015e350 0x7f68b015adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68b00b7200 0x7f68b01f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 --2- 172.21.15.64:0/1283167433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68b0071280 0x7f68b00ca530 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.039+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 >> 172.21.15.64:0/1283167433 conn(0x7f68b00d0e70 msgr2=0x7f68b00740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:53.040 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.040+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 shutdown_connections 2024-03-07T07:19:53.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.040+0000 7f68b54a1640 1 -- 172.21.15.64:0/1283167433 wait complete. 2024-03-07T07:19:53.050 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.154+0000 7f96ed89c640 1 Processor -- start 2024-03-07T07:19:53.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.154+0000 7f96ed89c640 1 -- start start 2024-03-07T07:19:53.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96ed89c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e8144870 0x7f96e815f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96ed89c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96e8071f80 0x7f96e80b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96ed89c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80d0030 con 0x7f96e80cf870 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96ed89c640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e8144c50 con 0x7f96e8071f80 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96ed89c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e815fa80 con 0x7f96e8144870 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e77fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96e8071f80 0x7f96e80b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e77fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f96e8071f80 0x7f96e80b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47100/0 (socket says 172.21.15.64:47100) 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e77fe640 1 -- 172.21.15.64:0/2292214828 learned_addr learned my addr 172.21.15.64:0/2292214828 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e67fc640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f96e80cf870 0x7f96e8071870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:37448/0 (socket says 172.21.15.64:37448) 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e6ffd640 1 --2- 172.21.15.64:0/2292214828 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e8144870 0x7f96e815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e77fe640 1 -- 172.21.15.64:0/2292214828 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e8144870 msgr2=0x7f96e815f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:53.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e77fe640 1 --2- 172.21.15.64:0/2292214828 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e8144870 0x7f96e815f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e5ffb640 1 -- 172.21.15.64:0/2292214828 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941896759 0 0) 0x7f96e80d0030 con 0x7f96e80cf870 2024-03-07T07:19:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e77fe640 1 -- 172.21.15.64:0/2292214828 >> v1:172.21.15.120:6789/0 conn(0x7f96e80cf870 legacy=0x7f96e8071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.155+0000 7f96e77fe640 1 -- 172.21.15.64:0/2292214828 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80d0310 con 0x7f96e8071f80 2024-03-07T07:19:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.156+0000 7f96e77fe640 1 --2- 172.21.15.64:0/2292214828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e80b4290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e80 tx=0x7f96d802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9124695c96732aa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.156+0000 7f96e5ffb640 1 -- 172.21.15.64:0/2292214828 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d802cb70 con 0x7f96e8071f80 2024-03-07T07:19:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.156+0000 7f96e5ffb640 1 -- 172.21.15.64:0/2292214828 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d802cd10 con 0x7f96e8071f80 2024-03-07T07:19:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.156+0000 7f96e5ffb640 1 -- 172.21.15.64:0/2292214828 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8034610 con 0x7f96e8071f80 2024-03-07T07:19:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 -- 172.21.15.64:0/2292214828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 msgr2=0x7f96e80b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 --2- 172.21.15.64:0/2292214828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e80b4290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e80 tx=0x7f96d802bfd0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 -- 172.21.15.64:0/2292214828 shutdown_connections 2024-03-07T07:19:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 --2- 172.21.15.64:0/2292214828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e80b4290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 --2- 172.21.15.64:0/2292214828 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e8144870 0x7f96e815f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 -- 172.21.15.64:0/2292214828 >> 172.21.15.64:0/2292214828 conn(0x7f96e80b64d0 msgr2=0x7f96e80b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 -- 172.21.15.64:0/2292214828 shutdown_connections 2024-03-07T07:19:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.157+0000 7f96ed89c640 1 -- 172.21.15.64:0/2292214828 wait complete. 2024-03-07T07:19:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.158+0000 7f96ed89c640 1 Processor -- start 2024-03-07T07:19:53.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.158+0000 7f96ed89c640 1 -- start start 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96ed89c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e81496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96ed89c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e80cf870 0x7f96e8140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96ed89c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96e8144870 0x7f96e81411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96ed89c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80d08d0 con 0x7f96e8071f80 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96ed89c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80d0c40 con 0x7f96e8144870 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96ed89c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80cfc70 con 0x7f96e80cf870 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e81496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e81496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47112/0 (socket says 172.21.15.64:47112) 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 -- 172.21.15.64:0/729101042 learned_addr learned my addr 172.21.15.64:0/729101042 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e77fe640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96e8144870 0x7f96e81411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e67fc640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e80cf870 0x7f96e8140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 -- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e80cf870 msgr2=0x7f96e8140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e80cf870 0x7f96e8140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 -- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96e8144870 msgr2=0x7f96e81411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96e8144870 0x7f96e81411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 -- 172.21.15.64:0/729101042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e813e4c0 con 0x7f96e8071f80 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e67fc640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e80cf870 0x7f96e8140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.159+0000 7f96e6ffd640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e81496e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e870 tx=0x7f96e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.160+0000 7f96dffff640 1 -- 172.21.15.64:0/729101042 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0017070 con 0x7f96e8071f80 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.160+0000 7f96dffff640 1 -- 172.21.15.64:0/729101042 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e000c940 con 0x7f96e8071f80 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.160+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e813e730 con 0x7f96e8071f80 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.160+0000 7f96dffff640 1 -- 172.21.15.64:0/729101042 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000cd30 con 0x7f96e8071f80 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.160+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e813ecf0 con 0x7f96e8071f80 2024-03-07T07:19:53.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.161+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e81440d0 con 0x7f96e8071f80 2024-03-07T07:19:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.161+0000 7f96dffff640 1 -- 172.21.15.64:0/729101042 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f96e001a070 con 0x7f96e8071f80 2024-03-07T07:19:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.161+0000 7f96dffff640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f96c806b4d0 0x7f96c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.161+0000 7f96dffff640 1 -- 172.21.15.64:0/729101042 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96e0089f80 con 0x7f96e8071f80 2024-03-07T07:19:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.162+0000 7f96e67fc640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f96c806b4d0 0x7f96c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.162+0000 7f96e67fc640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f96c806b4d0 0x7f96c806d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f96e80b2680 tx=0x7f96d0007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.166+0000 7f96dffff640 1 -- 172.21.15.64:0/729101042 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96e81440d0 con 0x7f96e8071f80 2024-03-07T07:19:53.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.345+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96e8062c90 con 0x7f96e8071f80 2024-03-07T07:19:53.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.346+0000 7f96dffff640 1 -- 172.21.15.64:0/729101042 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96e0058840 con 0x7f96e8071f80 2024-03-07T07:19:53.346 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f96c806b4d0 msgr2=0x7f96c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f96c806b4d0 0x7f96c806d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f96e80b2680 tx=0x7f96d0007c10 comp rx=0 tx=0).stop 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 msgr2=0x7f96e81496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e81496e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e870 tx=0x7f96e000ed30 comp rx=0 tx=0).stop 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 shutdown_connections 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f96c806b4d0 0x7f96c806d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96e8144870 0x7f96e81411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96e80cf870 0x7f96e8140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 --2- 172.21.15.64:0/729101042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8071f80 0x7f96e81496e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.347+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 >> 172.21.15.64:0/729101042 conn(0x7f96e80b64d0 msgr2=0x7f96e815f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:53.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.348+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 shutdown_connections 2024-03-07T07:19:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.348+0000 7f96ed89c640 1 -- 172.21.15.64:0/729101042 wait complete. 2024-03-07T07:19:53.358 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:53.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.462+0000 7fb55411d640 1 Processor -- start 2024-03-07T07:19:53.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.462+0000 7fb55411d640 1 -- start start 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.463+0000 7fb55411d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb54c09b0e0 0x7fb54c0b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.463+0000 7fb55411d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54c06d280 con 0x7fb54c0b1040 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.463+0000 7fb55411d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54c06c400 con 0x7fb54c161990 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.463+0000 7fb55411d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54c0b1770 con 0x7fb54c09b0e0 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.463+0000 7fb5526ba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fb54c0b1040 0x7fb54c0cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45354/0 (socket says 172.21.15.64:45354) 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.463+0000 7fb5526ba640 1 -- 172.21.15.64:0/1389775550 learned_addr learned my addr 172.21.15.64:0/1389775550 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.463+0000 7fb5516b8640 1 --2- 172.21.15.64:0/1389775550 >> v2:172.21.15.120:3300/0 conn(0x7fb54c09b0e0 0x7fb54c0b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.464+0000 7fb550eb7640 1 -- 172.21.15.64:0/1389775550 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780429782 0 0) 0x7fb54c06d280 con 0x7fb54c0b1040 2024-03-07T07:19:53.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.464+0000 7fb550eb7640 1 -- 172.21.15.64:0/1389775550 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb534003520 con 0x7fb54c0b1040 2024-03-07T07:19:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.464+0000 7fb550eb7640 1 -- 172.21.15.64:0/1389775550 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666726868 0 0) 0x7fb54c06c400 con 0x7fb54c161990 2024-03-07T07:19:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.464+0000 7fb550eb7640 1 -- 172.21.15.64:0/1389775550 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb54c06d280 con 0x7fb54c161990 2024-03-07T07:19:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.464+0000 7fb5516b8640 1 -- 172.21.15.64:0/1389775550 >> v1:172.21.15.120:6789/0 conn(0x7fb54c161990 legacy=0x7fb54c09a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.464+0000 7fb5516b8640 1 -- 172.21.15.64:0/1389775550 >> v1:172.21.15.64:6789/0 conn(0x7fb54c0b1040 legacy=0x7fb54c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.464+0000 7fb5516b8640 1 -- 172.21.15.64:0/1389775550 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c06c6e0 con 0x7fb54c09b0e0 2024-03-07T07:19:53.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb5516b8640 1 --2- 172.21.15.64:0/1389775550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c09b0e0 0x7fb54c0b46b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5400097a0 tx=0x7fb54002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd6439df88c273a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb550eb7640 1 -- 172.21.15.64:0/1389775550 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540035020 con 0x7fb54c09b0e0 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb550eb7640 1 -- 172.21.15.64:0/1389775550 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb540031830 con 0x7fb54c09b0e0 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb550eb7640 1 -- 172.21.15.64:0/1389775550 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540031be0 con 0x7fb54c09b0e0 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb55411d640 1 -- 172.21.15.64:0/1389775550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c09b0e0 msgr2=0x7fb54c0b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb55411d640 1 --2- 172.21.15.64:0/1389775550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c09b0e0 0x7fb54c0b46b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5400097a0 tx=0x7fb54002b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb55411d640 1 -- 172.21.15.64:0/1389775550 shutdown_connections 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb55411d640 1 --2- 172.21.15.64:0/1389775550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c09b0e0 0x7fb54c0b46b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb55411d640 1 -- 172.21.15.64:0/1389775550 >> 172.21.15.64:0/1389775550 conn(0x7fb54c0b3b90 msgr2=0x7fb54c0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb55411d640 1 -- 172.21.15.64:0/1389775550 shutdown_connections 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.465+0000 7fb55411d640 1 -- 172.21.15.64:0/1389775550 wait complete. 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 Processor -- start 2024-03-07T07:19:53.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 -- start start 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb54c09b0e0 0x7fb54c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c0b1040 0x7fb54c0b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb54c161990 0x7fb54c0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0d0770 con 0x7fb54c161990 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0d05f0 con 0x7fb54c0b1040 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb55411d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0d08f0 con 0x7fb54c09b0e0 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb5526ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb54c161990 0x7fb54c0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb5526ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb54c161990 0x7fb54c0b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47118/0 (socket says 172.21.15.64:47118) 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb5526ba640 1 -- 172.21.15.64:0/1573336438 learned_addr learned my addr 172.21.15.64:0/1573336438 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb5516b8640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c0b1040 0x7fb54c0b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.467+0000 7fb551eb9640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb54c09b0e0 0x7fb54c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb5526ba640 1 -- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb54c09b0e0 msgr2=0x7fb54c15b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb5526ba640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb54c09b0e0 0x7fb54c15b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb5526ba640 1 -- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c0b1040 msgr2=0x7fb54c0b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb5526ba640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c0b1040 0x7fb54c0b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb5526ba640 1 -- 172.21.15.64:0/1573336438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c15be70 con 0x7fb54c161990 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb5516b8640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c0b1040 0x7fb54c0b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb5526ba640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb54c161990 0x7fb54c0b71a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c440 tx=0x7fb54400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb54a7fc640 1 -- 172.21.15.64:0/1573336438 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544017070 con 0x7fb54c161990 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb54a7fc640 1 -- 172.21.15.64:0/1573336438 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544005ce0 con 0x7fb54c161990 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54c15ded0 con 0x7fb54c161990 2024-03-07T07:19:53.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb54a7fc640 1 -- 172.21.15.64:0/1573336438 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544007270 con 0x7fb54c161990 2024-03-07T07:19:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.468+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54c15e490 con 0x7fb54c161990 2024-03-07T07:19:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.469+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54c062c90 con 0x7fb54c161990 2024-03-07T07:19:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.469+0000 7fb54a7fc640 1 -- 172.21.15.64:0/1573336438 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb54401a070 con 0x7fb54c161990 2024-03-07T07:19:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.470+0000 7fb54a7fc640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb53806b4d0 0x7fb53806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.470+0000 7fb54a7fc640 1 -- 172.21.15.64:0/1573336438 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5440893e0 con 0x7fb54c161990 2024-03-07T07:19:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.470+0000 7fb551eb9640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb53806b4d0 0x7fb53806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.471+0000 7fb551eb9640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb53806b4d0 0x7fb53806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb53c001e00 tx=0x7fb53c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.474+0000 7fb54a7fc640 1 -- 172.21.15.64:0/1573336438 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb544057c10 con 0x7fb54c161990 2024-03-07T07:19:53.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.653+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb54c0b46d0 con 0x7fb54c161990 2024-03-07T07:19:53.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.654+0000 7fb54a7fc640 1 -- 172.21.15.64:0/1573336438 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb54405bcb0 con 0x7fb54c161990 2024-03-07T07:19:53.654 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:53.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb53806b4d0 msgr2=0x7fb53806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb53806b4d0 0x7fb53806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb53c001e00 tx=0x7fb53c009040 comp rx=0 tx=0).stop 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb54c161990 msgr2=0x7fb54c0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb54c161990 0x7fb54c0b71a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c440 tx=0x7fb54400c900 comp rx=0 tx=0).stop 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 shutdown_connections 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb53806b4d0 0x7fb53806d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb54c161990 0x7fb54c0b71a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb54c0b1040 0x7fb54c0b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 --2- 172.21.15.64:0/1573336438 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb54c09b0e0 0x7fb54c15b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 >> 172.21.15.64:0/1573336438 conn(0x7fb54c0b3b90 msgr2=0x7fb54c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 shutdown_connections 2024-03-07T07:19:53.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.656+0000 7fb55411d640 1 -- 172.21.15.64:0/1573336438 wait complete. 2024-03-07T07:19:53.666 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T07:19:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.769+0000 7f23de71d640 1 Processor -- start 2024-03-07T07:19:53.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.769+0000 7f23de71d640 1 -- start start 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23de71d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23d80b4f10 0x7f23d815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23de71d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b7360 0x7f23d80b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23de71d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d815d430 con 0x7f23d815fa60 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23de71d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d80d0970 con 0x7f23d80b4f10 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23de71d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80b5830 con 0x7f23d80b7360 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23d80b4f10 0x7f23d815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d7fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23d80b4f10 0x7f23d815cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47132/0 (socket says 172.21.15.64:47132) 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23dccba640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f23d815fa60 0x7f23d815ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:37464/0 (socket says 172.21.15.64:37464) 2024-03-07T07:19:53.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d7fff640 1 -- 172.21.15.64:0/2495575667 learned_addr learned my addr 172.21.15.64:0/2495575667 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d77fe640 1 --2- 172.21.15.64:0/2495575667 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b7360 0x7f23d80b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d6ffd640 1 -- 172.21.15.64:0/2495575667 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373772705 0 0) 0x7f23d0001490 con 0x7f23d815fa60 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d6ffd640 1 -- 172.21.15.64:0/2495575667 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c0003520 con 0x7f23d815fa60 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d7fff640 1 -- 172.21.15.64:0/2495575667 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b7360 msgr2=0x7f23d80b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d7fff640 1 --2- 172.21.15.64:0/2495575667 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b7360 0x7f23d80b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d7fff640 1 -- 172.21.15.64:0/2495575667 >> v1:172.21.15.120:6789/0 conn(0x7f23d815fa60 legacy=0x7f23d815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d7fff640 1 -- 172.21.15.64:0/2495575667 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d80d0bc0 con 0x7f23d80b4f10 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.770+0000 7f23d77fe640 1 --2- 172.21.15.64:0/2495575667 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b7360 0x7f23d80b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.771+0000 7f23d7fff640 1 --2- 172.21.15.64:0/2495575667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d80b4f10 0x7f23d815cef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009870 tx=0x7f23cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d565723298ec7b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.771+0000 7f23d6ffd640 1 -- 172.21.15.64:0/2495575667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc034070 con 0x7f23d80b4f10 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.771+0000 7f23d6ffd640 1 -- 172.21.15.64:0/2495575667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc030840 con 0x7f23d80b4f10 2024-03-07T07:19:53.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.771+0000 7f23d6ffd640 1 -- 172.21.15.64:0/2495575667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc030c10 con 0x7f23d80b4f10 2024-03-07T07:19:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 -- 172.21.15.64:0/2495575667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d80b4f10 msgr2=0x7f23d815cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 --2- 172.21.15.64:0/2495575667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d80b4f10 0x7f23d815cef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009870 tx=0x7f23cc02b980 comp rx=0 tx=0).stop 2024-03-07T07:19:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 -- 172.21.15.64:0/2495575667 shutdown_connections 2024-03-07T07:19:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 --2- 172.21.15.64:0/2495575667 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b7360 0x7f23d80b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 --2- 172.21.15.64:0/2495575667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d80b4f10 0x7f23d815cef0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 -- 172.21.15.64:0/2495575667 >> 172.21.15.64:0/2495575667 conn(0x7f23d806b480 msgr2=0x7f23d806b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:53.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 -- 172.21.15.64:0/2495575667 shutdown_connections 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.772+0000 7f23de71d640 1 -- 172.21.15.64:0/2495575667 wait complete. 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.773+0000 7f23de71d640 1 Processor -- start 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.773+0000 7f23de71d640 1 -- start start 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23de71d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b4f10 0x7f23d8151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23de71d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23d80b7360 0x7f23d814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23de71d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d815fa60 0x7f23d8153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23de71d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80d1460 con 0x7f23d815fa60 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23de71d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80d1180 con 0x7f23d80b7360 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23de71d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80d1740 con 0x7f23d80b4f10 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d815fa60 0x7f23d8153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23d77fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23d80b7360 0x7f23d814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d815fa60 0x7f23d8153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47140/0 (socket says 172.21.15.64:47140) 2024-03-07T07:19:53.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 -- 172.21.15.64:0/889937209 learned_addr learned my addr 172.21.15.64:0/889937209 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23d7fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b4f10 0x7f23d8151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 -- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b4f10 msgr2=0x7f23d8151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b4f10 0x7f23d8151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 -- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23d80b7360 msgr2=0x7f23d814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23d80b7360 0x7f23d814f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 -- 172.21.15.64:0/889937209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8151f40 con 0x7f23d815fa60 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23d77fe640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23d80b7360 0x7f23d814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23dccba640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d815fa60 0x7f23d8153b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c7c0 tx=0x7f23d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23d4ff9640 1 -- 172.21.15.64:0/889937209 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0017070 con 0x7f23d815fa60 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23d4ff9640 1 -- 172.21.15.64:0/889937209 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d000a640 con 0x7f23d815fa60 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d814fe20 con 0x7f23d815fa60 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.774+0000 7f23d4ff9640 1 -- 172.21.15.64:0/889937209 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d000aa70 con 0x7f23d815fa60 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.775+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d81503e0 con 0x7f23d815fa60 2024-03-07T07:19:53.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.775+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d80d1e70 con 0x7f23d815fa60 2024-03-07T07:19:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.776+0000 7f23d4ff9640 1 -- 172.21.15.64:0/889937209 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f23d001a070 con 0x7f23d815fa60 2024-03-07T07:19:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.776+0000 7f23d4ff9640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.776+0000 7f23d4ff9640 1 -- 172.21.15.64:0/889937209 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23d0089860 con 0x7f23d815fa60 2024-03-07T07:19:53.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.776+0000 7f23d7fff640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:53.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.777+0000 7f23d7fff640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f23cc036000 tx=0x7f23cc03b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:53.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.780+0000 7f23d4ff9640 1 -- 172.21.15.64:0/889937209 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f23d80d1e70 con 0x7f23d815fa60 2024-03-07T07:19:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.973+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23d806b860 con 0x7f23d815fa60 2024-03-07T07:19:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.973+0000 7f23d4ff9640 1 -- 172.21.15.64:0/889937209 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f23d0089030 con 0x7f23d815fa60 2024-03-07T07:19:53.973 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 msgr2=0x7f23c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f23cc036000 tx=0x7f23cc03b040 comp rx=0 tx=0).stop 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d815fa60 msgr2=0x7f23d8153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d815fa60 0x7f23d8153b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c7c0 tx=0x7f23d000cc80 comp rx=0 tx=0).stop 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 shutdown_connections 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23d815fa60 0x7f23d8153b10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23d80b7360 0x7f23d814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 --2- 172.21.15.64:0/889937209 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23d80b4f10 0x7f23d8151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 >> 172.21.15.64:0/889937209 conn(0x7f23d806b480 msgr2=0x7f23d815ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 shutdown_connections 2024-03-07T07:19:53.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:53.975+0000 7f23de71d640 1 -- 172.21.15.64:0/889937209 wait complete. 2024-03-07T07:19:53.986 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-07T07:19:53.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:53.986 DEBUG:teuthology.orchestra.run.smithi120:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-07T07:19:54.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:19:54.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:19:54.015 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-07T07:19:54.071 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-07T07:19:54.072 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.174+0000 7f85567de640 1 Processor -- start 2024-03-07T07:19:54.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.174+0000 7f85567de640 1 -- start start 2024-03-07T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f85567de640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85500d3ba0 0x7f855015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f85567de640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8550141c60 con 0x7f85501416d0 2024-03-07T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f85567de640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855015ecd0 con 0x7f8550162030 2024-03-07T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f85567de640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500b44b0 con 0x7f85500d3ba0 2024-03-07T07:19:54.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f854f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f85501416d0 0x7f85500d3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45370/0 (socket says 172.21.15.64:45370) 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f854f7fe640 1 -- 172.21.15.64:0/364701614 learned_addr learned my addr 172.21.15.64:0/364701614 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f8554d7b640 1 --2- 172.21.15.64:0/364701614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85500d3ba0 0x7f855015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f854effd640 1 -- 172.21.15.64:0/364701614 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066197143 0 0) 0x7f8550141c60 con 0x7f85501416d0 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f854effd640 1 -- 172.21.15.64:0/364701614 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8538003520 con 0x7f85501416d0 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f854effd640 1 -- 172.21.15.64:0/364701614 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076518693 0 0) 0x7f855015ecd0 con 0x7f8550162030 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f854effd640 1 -- 172.21.15.64:0/364701614 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8550141c60 con 0x7f8550162030 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f8554d7b640 1 -- 172.21.15.64:0/364701614 >> v1:172.21.15.120:6789/0 conn(0x7f8550162030 legacy=0x7f8550162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f8554d7b640 1 -- 172.21.15.64:0/364701614 >> v1:172.21.15.64:6789/0 conn(0x7f85501416d0 legacy=0x7f85500d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.175+0000 7f8554d7b640 1 -- 172.21.15.64:0/364701614 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855015efb0 con 0x7f85500d3ba0 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.176+0000 7f8554d7b640 1 --2- 172.21.15.64:0/364701614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85500d3ba0 0x7f855015e680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8548009dc0 tx=0x7f854802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8917d847549331b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.176+0000 7f854effd640 1 -- 172.21.15.64:0/364701614 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548039070 con 0x7f85500d3ba0 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.176+0000 7f854effd640 1 -- 172.21.15.64:0/364701614 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854802ccd0 con 0x7f85500d3ba0 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.176+0000 7f854effd640 1 -- 172.21.15.64:0/364701614 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548034640 con 0x7f85500d3ba0 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.177+0000 7f85567de640 1 -- 172.21.15.64:0/364701614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85500d3ba0 msgr2=0x7f855015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.177+0000 7f85567de640 1 --2- 172.21.15.64:0/364701614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85500d3ba0 0x7f855015e680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8548009dc0 tx=0x7f854802bf20 comp rx=0 tx=0).stop 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.177+0000 7f85567de640 1 -- 172.21.15.64:0/364701614 shutdown_connections 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.177+0000 7f85567de640 1 --2- 172.21.15.64:0/364701614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85500d3ba0 0x7f855015e680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.177+0000 7f85567de640 1 -- 172.21.15.64:0/364701614 >> 172.21.15.64:0/364701614 conn(0x7f8550070400 msgr2=0x7f85500d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.177+0000 7f85567de640 1 -- 172.21.15.64:0/364701614 shutdown_connections 2024-03-07T07:19:54.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.177+0000 7f85567de640 1 -- 172.21.15.64:0/364701614 wait complete. 2024-03-07T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.178+0000 7f85567de640 1 Processor -- start 2024-03-07T07:19:54.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.178+0000 7f85567de640 1 -- start start 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85500d3ba0 0x7f85501540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85501416d0 0x7f855015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8550162030 0x7f855015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855015f570 con 0x7f8550162030 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550162b20 con 0x7f85500d3ba0 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500b4940 con 0x7f85501416d0 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8550162030 0x7f855015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8550162030 0x7f855015afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47142/0 (socket says 172.21.15.64:47142) 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 -- 172.21.15.64:0/3856694494 learned_addr learned my addr 172.21.15.64:0/3856694494 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f854ffff640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85500d3ba0 0x7f85501540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f854f7fe640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85501416d0 0x7f855015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 -- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85501416d0 msgr2=0x7f855015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85501416d0 0x7f855015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 -- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85500d3ba0 msgr2=0x7f85501540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85500d3ba0 0x7f85501540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 -- 172.21.15.64:0/3856694494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855015e420 con 0x7f8550162030 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f8554d7b640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8550162030 0x7f855015afe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ee0 tx=0x7f8548004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f854cff9640 1 -- 172.21.15.64:0/3856694494 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548042020 con 0x7f8550162030 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f854cff9640 1 -- 172.21.15.64:0/3856694494 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854803a040 con 0x7f8550162030 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855015db80 con 0x7f8550162030 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f854cff9640 1 -- 172.21.15.64:0/3856694494 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548039070 con 0x7f8550162030 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.179+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855015e140 con 0x7f8550162030 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.180+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8550062c90 con 0x7f8550162030 2024-03-07T07:19:54.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.181+0000 7f854cff9640 1 -- 172.21.15.64:0/3856694494 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8548045020 con 0x7f8550162030 2024-03-07T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.181+0000 7f854cff9640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f853c06b4d0 0x7f853c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.182+0000 7f854cff9640 1 -- 172.21.15.64:0/3856694494 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85480ac220 con 0x7f8550162030 2024-03-07T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.182+0000 7f854ffff640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f853c06b4d0 0x7f853c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.182+0000 7f854ffff640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f853c06b4d0 0x7f853c06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8540007fd0 tx=0x7f8540009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.186+0000 7f854cff9640 1 -- 172.21.15.64:0/3856694494 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f854804c030 con 0x7f8550162030 2024-03-07T07:19:54.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.407+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85500008d0 con 0x7f8550162030 2024-03-07T07:19:54.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.407+0000 7f854cff9640 1 -- 172.21.15.64:0/3856694494 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7f854807aae0 con 0x7f8550162030 2024-03-07T07:19:54.407 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:54.407 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:54.408 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f853c06b4d0 msgr2=0x7f853c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f853c06b4d0 0x7f853c06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8540007fd0 tx=0x7f8540009040 comp rx=0 tx=0).stop 2024-03-07T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8550162030 msgr2=0x7f855015afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8550162030 0x7f855015afe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8548009ee0 tx=0x7f8548004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 shutdown_connections 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f853c06b4d0 0x7f853c06d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8550162030 0x7f855015afe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f85501416d0 0x7f855015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 --2- 172.21.15.64:0/3856694494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85500d3ba0 0x7f85501540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 >> 172.21.15.64:0/3856694494 conn(0x7f8550070400 msgr2=0x7f855015fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 shutdown_connections 2024-03-07T07:19:54.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.409+0000 7f85567de640 1 -- 172.21.15.64:0/3856694494 wait complete. 2024-03-07T07:19:54.421 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.522+0000 7f897ebc9640 1 Processor -- start 2024-03-07T07:19:54.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.522+0000 7f897ebc9640 1 -- start start 2024-03-07T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f897ebc9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89780d10a0 0x7f897815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f897ebc9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8978071c50 con 0x7f8978072cf0 2024-03-07T07:19:54.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f897ebc9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8978071f30 con 0x7f897815f170 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f897ebc9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897806c3a0 con 0x7f89780d10a0 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f8977fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89780d10a0 0x7f897815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f8977fff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f89780d10a0 0x7f897815e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47154/0 (socket says 172.21.15.64:47154) 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f8977fff640 1 -- 172.21.15.64:0/2120525902 learned_addr learned my addr 172.21.15.64:0/2120525902 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f897d166640 1 --1- 172.21.15.64:0/2120525902 >> v1:172.21.15.64:6789/0 conn(0x7f8978072cf0 0x7f89781470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45376/0 (socket says 172.21.15.64:45376) 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f8977fff640 1 -- 172.21.15.64:0/2120525902 >> v1:172.21.15.120:6789/0 conn(0x7f897815f170 legacy=0x7f89780d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f8977fff640 1 -- 172.21.15.64:0/2120525902 >> v1:172.21.15.64:6789/0 conn(0x7f8978072cf0 legacy=0x7f89781470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.523+0000 7f8977fff640 1 -- 172.21.15.64:0/2120525902 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8978072210 con 0x7f89780d10a0 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.524+0000 7f8977fff640 1 --2- 172.21.15.64:0/2120525902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89780d10a0 0x7f897815e650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8968009870 tx=0x7f896802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1666fe232e0c714 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.524+0000 7f89777fe640 1 -- 172.21.15.64:0/2120525902 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896802f840 con 0x7f89780d10a0 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.524+0000 7f89777fe640 1 -- 172.21.15.64:0/2120525902 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896802f9e0 con 0x7f89780d10a0 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.524+0000 7f89777fe640 1 -- 172.21.15.64:0/2120525902 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896802fdb0 con 0x7f89780d10a0 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.525+0000 7f897ebc9640 1 -- 172.21.15.64:0/2120525902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89780d10a0 msgr2=0x7f897815e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.525+0000 7f897ebc9640 1 --2- 172.21.15.64:0/2120525902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89780d10a0 0x7f897815e650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8968009870 tx=0x7f896802ba20 comp rx=0 tx=0).stop 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.525+0000 7f897ebc9640 1 -- 172.21.15.64:0/2120525902 shutdown_connections 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.525+0000 7f897ebc9640 1 --2- 172.21.15.64:0/2120525902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89780d10a0 0x7f897815e650 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.525+0000 7f897ebc9640 1 -- 172.21.15.64:0/2120525902 >> 172.21.15.64:0/2120525902 conn(0x7f897806b480 msgr2=0x7f897806b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.525+0000 7f897ebc9640 1 -- 172.21.15.64:0/2120525902 shutdown_connections 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.525+0000 7f897ebc9640 1 -- 172.21.15.64:0/2120525902 wait complete. 2024-03-07T07:19:54.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.526+0000 7f897ebc9640 1 Processor -- start 2024-03-07T07:19:54.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.526+0000 7f897ebc9640 1 -- start start 2024-03-07T07:19:54.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897ebc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8978072cf0 0x7f89781e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897ebc9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89780d10a0 0x7f897815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897ebc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897815f170 0x7f89780b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897ebc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978160560 con 0x7f897815f170 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897ebc9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89780727d0 con 0x7f8978072cf0 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897ebc9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8978071550 con 0x7f89780d10a0 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897815f170 0x7f89780b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897815f170 0x7f89780b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47156/0 (socket says 172.21.15.64:47156) 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 -- 172.21.15.64:0/1868292579 learned_addr learned my addr 172.21.15.64:0/1868292579 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897c965640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8978072cf0 0x7f89781e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f8977fff640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89780d10a0 0x7f897815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 -- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89780d10a0 msgr2=0x7f897815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89780d10a0 0x7f897815a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 -- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8978072cf0 msgr2=0x7f89781e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8978072cf0 0x7f89781e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 -- 172.21.15.64:0/1868292579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897815a9e0 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897d166640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897815f170 0x7f89780b5df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f896000c430 tx=0x7f896000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f89757fa640 1 -- 172.21.15.64:0/1868292579 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600074e0 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f89757fa640 1 -- 172.21.15.64:0/1868292579 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8960007680 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897815af70 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.527+0000 7f89757fa640 1 -- 172.21.15.64:0/1868292579 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600049e0 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.528+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897815b530 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.529+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8978062c90 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.529+0000 7f89757fa640 1 -- 172.21.15.64:0/1868292579 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f896001a020 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.529+0000 7f89757fa640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f895806b4d0 0x7f895806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.529+0000 7f89757fa640 1 -- 172.21.15.64:0/1868292579 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8960089870 con 0x7f897815f170 2024-03-07T07:19:54.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.529+0000 7f897c965640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f895806b4d0 0x7f895806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.530+0000 7f897c965640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f895806b4d0 0x7f895806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8970001e70 tx=0x7f8970009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.533+0000 7f89757fa640 1 -- 172.21.15.64:0/1868292579 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8960058080 con 0x7f897815f170 2024-03-07T07:19:54.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.715+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8978147d70 con 0x7f897815f170 2024-03-07T07:19:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.717+0000 7f89757fa640 1 -- 172.21.15.64:0/1868292579 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f896005c120 con 0x7f897815f170 2024-03-07T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:54.717 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f895806b4d0 msgr2=0x7f895806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f895806b4d0 0x7f895806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8970001e70 tx=0x7f8970009040 comp rx=0 tx=0).stop 2024-03-07T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897815f170 msgr2=0x7f89780b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897815f170 0x7f89780b5df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f896000c430 tx=0x7f896000c8f0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 shutdown_connections 2024-03-07T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f895806b4d0 0x7f895806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f897815f170 0x7f89780b5df0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89780d10a0 0x7f897815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 --2- 172.21.15.64:0/1868292579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8978072cf0 0x7f89781e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 >> 172.21.15.64:0/1868292579 conn(0x7f897806b480 msgr2=0x7f8978162580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 shutdown_connections 2024-03-07T07:19:54.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.719+0000 7f897ebc9640 1 -- 172.21.15.64:0/1868292579 wait complete. 2024-03-07T07:19:54.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:19:14.236451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:19:14.610190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:19:54.730 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.834+0000 7f9b99fa7640 1 Processor -- start 2024-03-07T07:19:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.834+0000 7f9b99fa7640 1 -- start start 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.834+0000 7f9b99fa7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9415f800 0x7f9b9409a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.834+0000 7f9b99fa7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940d1090 con 0x7f9b9415fbe0 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.834+0000 7f9b99fa7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940d1370 con 0x7f9b940b0fb0 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.834+0000 7f9b99fa7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940d1650 con 0x7f9b9415f800 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b92ffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9b9415fbe0 0x7f9b940cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45384/0 (socket says 172.21.15.64:45384) 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b92ffd640 1 -- 172.21.15.64:0/529088781 learned_addr learned my addr 172.21.15.64:0/529088781 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b937fe640 1 --2- 172.21.15.64:0/529088781 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9415f800 0x7f9b9409a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b927fc640 1 -- 172.21.15.64:0/529088781 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254091080 0 0) 0x7f9b940d1090 con 0x7f9b9415fbe0 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b927fc640 1 -- 172.21.15.64:0/529088781 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b80003520 con 0x7f9b9415fbe0 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b927fc640 1 -- 172.21.15.64:0/529088781 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256605524 0 0) 0x7f9b940d1370 con 0x7f9b940b0fb0 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b937fe640 1 -- 172.21.15.64:0/529088781 >> v1:172.21.15.120:6789/0 conn(0x7f9b940b0fb0 legacy=0x7f9b940d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b937fe640 1 -- 172.21.15.64:0/529088781 >> v1:172.21.15.64:6789/0 conn(0x7f9b9415fbe0 legacy=0x7f9b940cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.835+0000 7f9b937fe640 1 -- 172.21.15.64:0/529088781 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940d1930 con 0x7f9b9415f800 2024-03-07T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b937fe640 1 --2- 172.21.15.64:0/529088781 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9415f800 0x7f9b9409a9d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097d0 tx=0x7f9b8402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec2318be37eb053 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b927fc640 1 -- 172.21.15.64:0/529088781 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84035020 con 0x7f9b9415f800 2024-03-07T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b927fc640 1 -- 172.21.15.64:0/529088781 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b84031890 con 0x7f9b9415f800 2024-03-07T07:19:54.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b927fc640 1 -- 172.21.15.64:0/529088781 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84031c60 con 0x7f9b9415f800 2024-03-07T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b99fa7640 1 -- 172.21.15.64:0/529088781 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9415f800 msgr2=0x7f9b9409a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/529088781 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9415f800 0x7f9b9409a9d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097d0 tx=0x7f9b8402b930 comp rx=0 tx=0).stop 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b99fa7640 1 -- 172.21.15.64:0/529088781 shutdown_connections 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/529088781 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9415f800 0x7f9b9409a9d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b99fa7640 1 -- 172.21.15.64:0/529088781 >> 172.21.15.64:0/529088781 conn(0x7f9b940b3b90 msgr2=0x7f9b940b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b99fa7640 1 -- 172.21.15.64:0/529088781 shutdown_connections 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.836+0000 7f9b99fa7640 1 -- 172.21.15.64:0/529088781 wait complete. 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.837+0000 7f9b99fa7640 1 Processor -- start 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.838+0000 7f9b99fa7640 1 -- start start 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.838+0000 7f9b99fa7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b940b0fb0 0x7f9b94154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.838+0000 7f9b99fa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b9415f800 0x7f9b9415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.838+0000 7f9b99fa7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9415fbe0 0x7f9b9415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.838+0000 7f9b99fa7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940d0d00 con 0x7f9b9415f800 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.838+0000 7f9b99fa7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940b15e0 con 0x7f9b9415fbe0 2024-03-07T07:19:54.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.838+0000 7f9b99fa7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940cdc60 con 0x7f9b940b0fb0 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b9415f800 0x7f9b9415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b9415f800 0x7f9b9415c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47166/0 (socket says 172.21.15.64:47166) 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 -- 172.21.15.64:0/2226585090 learned_addr learned my addr 172.21.15.64:0/2226585090 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b93fff640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9415fbe0 0x7f9b9415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b937fe640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b940b0fb0 0x7f9b94154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 -- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b940b0fb0 msgr2=0x7f9b94154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b940b0fb0 0x7f9b94154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 -- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9415fbe0 msgr2=0x7f9b9415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9415fbe0 0x7f9b9415cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 -- 172.21.15.64:0/2226585090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94155370 con 0x7f9b9415f800 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b92ffd640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b9415f800 0x7f9b9415c9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c390 tx=0x7f9b7c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b8bfff640 1 -- 172.21.15.64:0/2226585090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c017070 con 0x7f9b9415f800 2024-03-07T07:19:54.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b8bfff640 1 -- 172.21.15.64:0/2226585090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c005ce0 con 0x7f9b9415f800 2024-03-07T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b940c5bb0 con 0x7f9b9415f800 2024-03-07T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b8bfff640 1 -- 172.21.15.64:0/2226585090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0072f0 con 0x7f9b9415f800 2024-03-07T07:19:54.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.839+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b940c5e50 con 0x7f9b9415f800 2024-03-07T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.840+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b94062c90 con 0x7f9b9415f800 2024-03-07T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.841+0000 7f9b8bfff640 1 -- 172.21.15.64:0/2226585090 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9b7c01a070 con 0x7f9b9415f800 2024-03-07T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.841+0000 7f9b8bfff640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7406b500 0x7f9b7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.841+0000 7f9b8bfff640 1 -- 172.21.15.64:0/2226585090 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b7c089660 con 0x7f9b9415f800 2024-03-07T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.841+0000 7f9b937fe640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7406b500 0x7f9b7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:54.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.842+0000 7f9b937fe640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7406b500 0x7f9b7406d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b84036000 tx=0x7f9b84008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:54.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:54.845+0000 7f9b8bfff640 1 -- 172.21.15.64:0/2226585090 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b7c057d80 con 0x7f9b9415f800 2024-03-07T07:19:55.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.027+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b940b0be0 con 0x7f9b9415f800 2024-03-07T07:19:55.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.028+0000 7f9b8bfff640 1 -- 172.21.15.64:0/2226585090 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b7c05be20 con 0x7f9b9415f800 2024-03-07T07:19:55.028 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7406b500 msgr2=0x7f9b7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7406b500 0x7f9b7406d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b84036000 tx=0x7f9b84008030 comp rx=0 tx=0).stop 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b9415f800 msgr2=0x7f9b9415c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b9415f800 0x7f9b9415c9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c390 tx=0x7f9b7c00c850 comp rx=0 tx=0).stop 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 shutdown_connections 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9b7406b500 0x7f9b7406d9c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b9415f800 0x7f9b9415c9f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9415fbe0 0x7f9b9415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 --2- 172.21.15.64:0/2226585090 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b940b0fb0 0x7f9b94154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.029+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 >> 172.21.15.64:0/2226585090 conn(0x7f9b940b3b90 msgr2=0x7f9b940afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:55.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.030+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 shutdown_connections 2024-03-07T07:19:55.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.030+0000 7f9b99fa7640 1 -- 172.21.15.64:0/2226585090 wait complete. 2024-03-07T07:19:55.041 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:19:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.145+0000 7f0e61ed4640 1 Processor -- start 2024-03-07T07:19:55.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.145+0000 7f0e61ed4640 1 -- start start 2024-03-07T07:19:55.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e61ed4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0e5c162110 0x7f0e5c1624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e61ed4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c070bc0 0x7f0e5c071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e61ed4640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e5c15e8d0 0x7f0e5c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e61ed4640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c162a30 con 0x7f0e5c15e8d0 2024-03-07T07:19:55.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e61ed4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c071610 con 0x7f0e5c162110 2024-03-07T07:19:55.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e61ed4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0718f0 con 0x7f0e5c070bc0 2024-03-07T07:19:55.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e5c15e8d0 0x7f0e5c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f0e5c15e8d0 0x7f0e5c15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47172/0 (socket says 172.21.15.64:47172) 2024-03-07T07:19:55.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 -- 172.21.15.64:0/1150266116 learned_addr learned my addr 172.21.15.64:0/1150266116 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:55.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5b7fe640 1 --2- 172.21.15.64:0/1150266116 >> v2:172.21.15.120:3300/0 conn(0x7f0e5c162110 0x7f0e5c1624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5affd640 1 --2- 172.21.15.64:0/1150266116 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c070bc0 0x7f0e5c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 -- 172.21.15.64:0/1150266116 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c070bc0 msgr2=0x7f0e5c071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:55.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 --2- 172.21.15.64:0/1150266116 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c070bc0 0x7f0e5c071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 -- 172.21.15.64:0/1150266116 >> v2:172.21.15.120:3300/0 conn(0x7f0e5c162110 msgr2=0x7f0e5c1624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:55.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 --2- 172.21.15.64:0/1150266116 >> v2:172.21.15.120:3300/0 conn(0x7f0e5c162110 0x7f0e5c1624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5bfff640 1 -- 172.21.15.64:0/1150266116 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c071bd0 con 0x7f0e5c15e8d0 2024-03-07T07:19:55.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.146+0000 7f0e5b7fe640 1 --2- 172.21.15.64:0/1150266116 >> v2:172.21.15.120:3300/0 conn(0x7f0e5c162110 0x7f0e5c1624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:55.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.147+0000 7f0e5bfff640 1 --2- 172.21.15.64:0/1150266116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c15e8d0 0x7f0e5c15ed60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c08d270 tx=0x7f0e5402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ade3491f95695c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.147+0000 7f0e5a7fc640 1 -- 172.21.15.64:0/1150266116 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5402ca70 con 0x7f0e5c15e8d0 2024-03-07T07:19:55.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.147+0000 7f0e5a7fc640 1 -- 172.21.15.64:0/1150266116 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5402cc10 con 0x7f0e5c15e8d0 2024-03-07T07:19:55.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.147+0000 7f0e5a7fc640 1 -- 172.21.15.64:0/1150266116 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540343f0 con 0x7f0e5c15e8d0 2024-03-07T07:19:55.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 -- 172.21.15.64:0/1150266116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c15e8d0 msgr2=0x7f0e5c15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/1150266116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c15e8d0 0x7f0e5c15ed60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c08d270 tx=0x7f0e5402bed0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 -- 172.21.15.64:0/1150266116 shutdown_connections 2024-03-07T07:19:55.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/1150266116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c15e8d0 0x7f0e5c15ed60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/1150266116 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c070bc0 0x7f0e5c071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/1150266116 >> v2:172.21.15.120:3300/0 conn(0x7f0e5c162110 0x7f0e5c1624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 -- 172.21.15.64:0/1150266116 >> 172.21.15.64:0/1150266116 conn(0x7f0e5c0d0e10 msgr2=0x7f0e5c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:55.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 -- 172.21.15.64:0/1150266116 shutdown_connections 2024-03-07T07:19:55.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.148+0000 7f0e61ed4640 1 -- 172.21.15.64:0/1150266116 wait complete. 2024-03-07T07:19:55.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.149+0000 7f0e61ed4640 1 Processor -- start 2024-03-07T07:19:55.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.149+0000 7f0e61ed4640 1 -- start start 2024-03-07T07:19:55.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e61ed4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e5c070bc0 0x7f0e5c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e61ed4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c15e8d0 0x7f0e5c1e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e61ed4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c162110 0x7f0e5c154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e61ed4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c162e00 con 0x7f0e5c162110 2024-03-07T07:19:55.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e61ed4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0d21d0 con 0x7f0e5c070bc0 2024-03-07T07:19:55.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e61ed4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c162f80 con 0x7f0e5c15e8d0 2024-03-07T07:19:55.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c162110 0x7f0e5c154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c162110 0x7f0e5c154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47180/0 (socket says 172.21.15.64:47180) 2024-03-07T07:19:55.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 -- 172.21.15.64:0/4034157390 learned_addr learned my addr 172.21.15.64:0/4034157390 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:55.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5b7fe640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e5c070bc0 0x7f0e5c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5affd640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c15e8d0 0x7f0e5c1e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 -- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c15e8d0 msgr2=0x7f0e5c1e8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c15e8d0 0x7f0e5c1e8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 -- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e5c070bc0 msgr2=0x7f0e5c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e5c070bc0 0x7f0e5c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5bfff640 1 -- 172.21.15.64:0/4034157390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c158f80 con 0x7f0e5c162110 2024-03-07T07:19:55.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.150+0000 7f0e5b7fe640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e5c070bc0 0x7f0e5c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:55.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.151+0000 7f0e5bfff640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c162110 0x7f0e5c154360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e54036040 tx=0x7f0e54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.151+0000 7f0e43fff640 1 -- 172.21.15.64:0/4034157390 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54043020 con 0x7f0e5c162110 2024-03-07T07:19:55.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.151+0000 7f0e43fff640 1 -- 172.21.15.64:0/4034157390 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5403e450 con 0x7f0e5c162110 2024-03-07T07:19:55.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.151+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c155080 con 0x7f0e5c162110 2024-03-07T07:19:55.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.151+0000 7f0e43fff640 1 -- 172.21.15.64:0/4034157390 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54009510 con 0x7f0e5c162110 2024-03-07T07:19:55.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.151+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c158af0 con 0x7f0e5c162110 2024-03-07T07:19:55.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.152+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c062c90 con 0x7f0e5c162110 2024-03-07T07:19:55.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.152+0000 7f0e43fff640 1 -- 172.21.15.64:0/4034157390 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e54046020 con 0x7f0e5c162110 2024-03-07T07:19:55.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.153+0000 7f0e43fff640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e4806b4d0 0x7f0e4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.153+0000 7f0e43fff640 1 -- 172.21.15.64:0/4034157390 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e540ac7a0 con 0x7f0e5c162110 2024-03-07T07:19:55.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.153+0000 7f0e5b7fe640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e4806b4d0 0x7f0e4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.153+0000 7f0e5b7fe640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e4806b4d0 0x7f0e4806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009560 tx=0x7f0e4c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.157+0000 7f0e43fff640 1 -- 172.21.15.64:0/4034157390 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e5404d030 con 0x7f0e5c162110 2024-03-07T07:19:55.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.338+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e5c070fa0 con 0x7f0e5c162110 2024-03-07T07:19:55.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.338+0000 7f0e43fff640 1 -- 172.21.15.64:0/4034157390 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e5407afd0 con 0x7f0e5c162110 2024-03-07T07:19:55.339 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:19:55.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e4806b4d0 msgr2=0x7f0e4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e4806b4d0 0x7f0e4806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009560 tx=0x7f0e4c009190 comp rx=0 tx=0).stop 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c162110 msgr2=0x7f0e5c154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c162110 0x7f0e5c154360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e54036040 tx=0x7f0e54004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 shutdown_connections 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f0e4806b4d0 0x7f0e4806d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e5c162110 0x7f0e5c154360 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e5c15e8d0 0x7f0e5c1e8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 --2- 172.21.15.64:0/4034157390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e5c070bc0 0x7f0e5c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 >> 172.21.15.64:0/4034157390 conn(0x7f0e5c0d0e10 msgr2=0x7f0e5c0b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 shutdown_connections 2024-03-07T07:19:55.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.340+0000 7f0e61ed4640 1 -- 172.21.15.64:0/4034157390 wait complete. 2024-03-07T07:19:55.351 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:19:55.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.455+0000 7f2c3ead0640 1 Processor -- start 2024-03-07T07:19:55.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.455+0000 7f2c3ead0640 1 -- start start 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3ead0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2c381629c0 0x7f2c380b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3ead0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c380b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3ead0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c380b34d0 con 0x7f2c3815dff0 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3ead0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c38161ee0 con 0x7f2c381629c0 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3ead0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c381621c0 con 0x7f2c3815e3d0 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c37fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2c381629c0 0x7f2c380b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3d06d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c380b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3c86c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f2c3815dff0 0x7f2c380b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:37504/0 (socket says 172.21.15.64:37504) 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.456+0000 7f2c3c86c640 1 -- 172.21.15.64:0/1888881201 learned_addr learned my addr 172.21.15.64:0/1888881201 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:55.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.457+0000 7f2c3d06d640 1 -- 172.21.15.64:0/1888881201 >> v2:172.21.15.120:3300/0 conn(0x7f2c381629c0 msgr2=0x7f2c380b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.457+0000 7f2c3d06d640 1 --2- 172.21.15.64:0/1888881201 >> v2:172.21.15.120:3300/0 conn(0x7f2c381629c0 0x7f2c380b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.457+0000 7f2c3d06d640 1 -- 172.21.15.64:0/1888881201 >> v1:172.21.15.120:6789/0 conn(0x7f2c3815dff0 legacy=0x7f2c380b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.457+0000 7f2c3d06d640 1 -- 172.21.15.64:0/1888881201 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c381624a0 con 0x7f2c3815e3d0 2024-03-07T07:19:55.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.457+0000 7f2c37fff640 1 --2- 172.21.15.64:0/1888881201 >> v2:172.21.15.120:3300/0 conn(0x7f2c381629c0 0x7f2c380b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3d06d640 1 --2- 172.21.15.64:0/1888881201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c380b4330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c280097a0 tx=0x7f2c2802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4361538b47e392e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c377fe640 1 -- 172.21.15.64:0/1888881201 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28035020 con 0x7f2c3815e3d0 2024-03-07T07:19:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c377fe640 1 -- 172.21.15.64:0/1888881201 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28031830 con 0x7f2c3815e3d0 2024-03-07T07:19:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c377fe640 1 -- 172.21.15.64:0/1888881201 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28031c00 con 0x7f2c3815e3d0 2024-03-07T07:19:55.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 -- 172.21.15.64:0/1888881201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 msgr2=0x7f2c380b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/1888881201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c380b4330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c280097a0 tx=0x7f2c2802b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 -- 172.21.15.64:0/1888881201 shutdown_connections 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/1888881201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c380b4330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/1888881201 >> v2:172.21.15.120:3300/0 conn(0x7f2c381629c0 0x7f2c380b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 -- 172.21.15.64:0/1888881201 >> 172.21.15.64:0/1888881201 conn(0x7f2c3806b040 msgr2=0x7f2c3806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 -- 172.21.15.64:0/1888881201 shutdown_connections 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.458+0000 7f2c3ead0640 1 -- 172.21.15.64:0/1888881201 wait complete. 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.459+0000 7f2c3ead0640 1 Processor -- start 2024-03-07T07:19:55.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.460+0000 7f2c3ead0640 1 -- start start 2024-03-07T07:19:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.460+0000 7f2c3ead0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3815dff0 0x7f2c38152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.460+0000 7f2c3ead0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c3814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.460+0000 7f2c3ead0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c381629c0 0x7f2c3815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.460+0000 7f2c3ead0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380b0100 con 0x7f2c381629c0 2024-03-07T07:19:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.460+0000 7f2c3ead0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380aff80 con 0x7f2c3815dff0 2024-03-07T07:19:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.460+0000 7f2c3ead0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380b4a00 con 0x7f2c3815e3d0 2024-03-07T07:19:55.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c381629c0 0x7f2c3815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c381629c0 0x7f2c3815a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47194/0 (socket says 172.21.15.64:47194) 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 -- 172.21.15.64:0/2535123916 learned_addr learned my addr 172.21.15.64:0/2535123916 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3c86c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3815dff0 0x7f2c38152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c37fff640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c3814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 -- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 msgr2=0x7f2c3814fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c3814fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 -- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3815dff0 msgr2=0x7f2c38152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3815dff0 0x7f2c38152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 -- 172.21.15.64:0/2535123916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c38151fe0 con 0x7f2c381629c0 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3d06d640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c381629c0 0x7f2c3815a620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c2802bee0 tx=0x7f2c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c357fa640 1 -- 172.21.15.64:0/2535123916 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28035040 con 0x7f2c381629c0 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c357fa640 1 -- 172.21.15.64:0/2535123916 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2803f070 con 0x7f2c381629c0 2024-03-07T07:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c357fa640 1 -- 172.21.15.64:0/2535123916 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2803ba80 con 0x7f2c381629c0 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c381522c0 con 0x7f2c381629c0 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.461+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c38153320 con 0x7f2c381629c0 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.462+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c38062c90 con 0x7f2c381629c0 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.463+0000 7f2c357fa640 1 -- 172.21.15.64:0/2535123916 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2c28038020 con 0x7f2c381629c0 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.463+0000 7f2c357fa640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c2406b4d0 0x7f2c2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.463+0000 7f2c357fa640 1 -- 172.21.15.64:0/2535123916 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c280ac030 con 0x7f2c381629c0 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.463+0000 7f2c3c86c640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c2406b4d0 0x7f2c2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.464+0000 7f2c3c86c640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c2406b4d0 0x7f2c2406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2c300012e0 tx=0x7f2c30001a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.467+0000 7f2c357fa640 1 -- 172.21.15.64:0/2535123916 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2c28031800 con 0x7f2c381629c0 2024-03-07T07:19:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.650+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c38157d90 con 0x7f2c381629c0 2024-03-07T07:19:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.650+0000 7f2c357fa640 1 -- 172.21.15.64:0/2535123916 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c2807a7d0 con 0x7f2c381629c0 2024-03-07T07:19:55.651 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c2406b4d0 msgr2=0x7f2c2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c2406b4d0 0x7f2c2406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2c300012e0 tx=0x7f2c30001a30 comp rx=0 tx=0).stop 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c381629c0 msgr2=0x7f2c3815a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c381629c0 0x7f2c3815a620 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c2802bee0 tx=0x7f2c280046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 shutdown_connections 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2c2406b4d0 0x7f2c2406d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c381629c0 0x7f2c3815a620 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2c3815e3d0 0x7f2c3814fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 --2- 172.21.15.64:0/2535123916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c3815dff0 0x7f2c38152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 >> 172.21.15.64:0/2535123916 conn(0x7f2c3806b040 msgr2=0x7f2c380b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:55.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 shutdown_connections 2024-03-07T07:19:55.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.653+0000 7f2c3ead0640 1 -- 172.21.15.64:0/2535123916 wait complete. 2024-03-07T07:19:55.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:55.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.764+0000 7fec26fa7640 1 Processor -- start 2024-03-07T07:19:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.764+0000 7fec26fa7640 1 -- start start 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec26fa7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec20145bf0 0x7fec200cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec26fa7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec2006bc20 0x7fec200b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec26fa7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec200d0aa0 con 0x7fec200d1860 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec26fa7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec200d13d0 con 0x7fec2006bc20 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec26fa7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec200d0d80 con 0x7fec20145bf0 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec25544640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec2006bc20 0x7fec200b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec25544640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fec2006bc20 0x7fec200b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47208/0 (socket says 172.21.15.64:47208) 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec25544640 1 -- 172.21.15.64:0/3053130396 learned_addr learned my addr 172.21.15.64:0/3053130396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.765+0000 7fec24d43640 1 --2- 172.21.15.64:0/3053130396 >> v2:172.21.15.120:3300/0 conn(0x7fec20145bf0 0x7fec200cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.766+0000 7fec25544640 1 -- 172.21.15.64:0/3053130396 >> v2:172.21.15.120:3300/0 conn(0x7fec20145bf0 msgr2=0x7fec200cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.766+0000 7fec25544640 1 --2- 172.21.15.64:0/3053130396 >> v2:172.21.15.120:3300/0 conn(0x7fec20145bf0 0x7fec200cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.766+0000 7fec25544640 1 -- 172.21.15.64:0/3053130396 >> v1:172.21.15.120:6789/0 conn(0x7fec200d1860 legacy=0x7fec2006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.766+0000 7fec25544640 1 -- 172.21.15.64:0/3053130396 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200d1060 con 0x7fec2006bc20 2024-03-07T07:19:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.766+0000 7fec25544640 1 --2- 172.21.15.64:0/3053130396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec2006bc20 0x7fec200b5160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fec2006c130 tx=0x7fec1002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2776c2acadf33724 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.766+0000 7fec1f7fe640 1 -- 172.21.15.64:0/3053130396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10034070 con 0x7fec2006bc20 2024-03-07T07:19:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec1f7fe640 1 -- 172.21.15.64:0/3053130396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10030860 con 0x7fec2006bc20 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec1f7fe640 1 -- 172.21.15.64:0/3053130396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10030c30 con 0x7fec2006bc20 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 -- 172.21.15.64:0/3053130396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec2006bc20 msgr2=0x7fec200b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 --2- 172.21.15.64:0/3053130396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec2006bc20 0x7fec200b5160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fec2006c130 tx=0x7fec1002b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 -- 172.21.15.64:0/3053130396 shutdown_connections 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 --2- 172.21.15.64:0/3053130396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec2006bc20 0x7fec200b5160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 --2- 172.21.15.64:0/3053130396 >> v2:172.21.15.120:3300/0 conn(0x7fec20145bf0 0x7fec200cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 -- 172.21.15.64:0/3053130396 >> 172.21.15.64:0/3053130396 conn(0x7fec2006d4c0 msgr2=0x7fec20066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 -- 172.21.15.64:0/3053130396 shutdown_connections 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.767+0000 7fec26fa7640 1 -- 172.21.15.64:0/3053130396 wait complete. 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.768+0000 7fec26fa7640 1 Processor -- start 2024-03-07T07:19:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.768+0000 7fec26fa7640 1 -- start start 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec26fa7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec2006bc20 0x7fec2015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec26fa7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec200d1860 0x7fec2013b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec26fa7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec20145bf0 0x7fec2013f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec26fa7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200cf130 con 0x7fec200d1860 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec26fa7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200cefb0 con 0x7fec20145bf0 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec26fa7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec200d06c0 con 0x7fec2006bc20 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec200d1860 0x7fec2013b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec200d1860 0x7fec2013b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47212/0 (socket says 172.21.15.64:47212) 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec1ffff640 1 -- 172.21.15.64:0/513140267 learned_addr learned my addr 172.21.15.64:0/513140267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec25544640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec20145bf0 0x7fec2013f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.769+0000 7fec24d43640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec2006bc20 0x7fec2015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1ffff640 1 -- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec2006bc20 msgr2=0x7fec2015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1ffff640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec2006bc20 0x7fec2015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1ffff640 1 -- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec20145bf0 msgr2=0x7fec2013f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1ffff640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec20145bf0 0x7fec2013f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1ffff640 1 -- 172.21.15.64:0/513140267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2013a8f0 con 0x7fec200d1860 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1ffff640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec200d1860 0x7fec2013b480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fec1400c400 tx=0x7fec1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1d7fa640 1 -- 172.21.15.64:0/513140267 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140052d0 con 0x7fec200d1860 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1d7fa640 1 -- 172.21.15.64:0/513140267 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec14005490 con 0x7fec200d1860 2024-03-07T07:19:55.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2013d420 con 0x7fec200d1860 2024-03-07T07:19:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec1d7fa640 1 -- 172.21.15.64:0/513140267 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14007260 con 0x7fec200d1860 2024-03-07T07:19:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.770+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2013d9e0 con 0x7fec200d1860 2024-03-07T07:19:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.771+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec20062c90 con 0x7fec200d1860 2024-03-07T07:19:55.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.772+0000 7fec1d7fa640 1 -- 172.21.15.64:0/513140267 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fec14007400 con 0x7fec200d1860 2024-03-07T07:19:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.772+0000 7fec1d7fa640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fec0c06b4d0 0x7fec0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.772+0000 7fec1d7fa640 1 -- 172.21.15.64:0/513140267 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec140896a0 con 0x7fec200d1860 2024-03-07T07:19:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.772+0000 7fec24d43640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fec0c06b4d0 0x7fec0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.773+0000 7fec24d43640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fec0c06b4d0 0x7fec0c06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec18011340 tx=0x7fec1800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:55.776+0000 7fec1d7fa640 1 -- 172.21.15.64:0/513140267 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fec14057ee0 con 0x7fec200d1860 2024-03-07T07:19:56.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.014+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec200b2110 con 0x7fec200d1860 2024-03-07T07:19:56.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.015+0000 7fec1d7fa640 1 -- 172.21.15.64:0/513140267 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7fec1405bf80 con 0x7fec200d1860 2024-03-07T07:19:56.015 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:56.015 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:56.016 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fec0c06b4d0 msgr2=0x7fec0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fec0c06b4d0 0x7fec0c06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec18011340 tx=0x7fec1800b040 comp rx=0 tx=0).stop 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec200d1860 msgr2=0x7fec2013b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec200d1860 0x7fec2013b480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fec1400c400 tx=0x7fec1400c8c0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 shutdown_connections 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fec0c06b4d0 0x7fec0c06d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec200d1860 0x7fec2013b480 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec20145bf0 0x7fec2013f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 --2- 172.21.15.64:0/513140267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec2006bc20 0x7fec2015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 >> 172.21.15.64:0/513140267 conn(0x7fec2006d4c0 msgr2=0x7fec201404b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:56.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 shutdown_connections 2024-03-07T07:19:56.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.017+0000 7fec26fa7640 1 -- 172.21.15.64:0/513140267 wait complete. 2024-03-07T07:19:56.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:56.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.129+0000 7fa040554640 1 Processor -- start 2024-03-07T07:19:56.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.129+0000 7fa040554640 1 -- start start 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.130+0000 7fa040554640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa0380d1860 0x7fa03806b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.130+0000 7fa040554640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0380d0aa0 con 0x7fa038145bf0 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.130+0000 7fa040554640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0380d0d80 con 0x7fa0380b7390 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.130+0000 7fa040554640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0380d13d0 con 0x7fa0380d1860 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.130+0000 7fa03e2f0640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa038145bf0 0x7fa0380cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45388/0 (socket says 172.21.15.64:45388) 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.131+0000 7fa03e2f0640 1 -- 172.21.15.64:0/3833019580 learned_addr learned my addr 172.21.15.64:0/3833019580 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.131+0000 7fa03daef640 1 --2- 172.21.15.64:0/3833019580 >> v2:172.21.15.64:3300/0 conn(0x7fa0380d1860 0x7fa03806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.131+0000 7fa03daef640 1 -- 172.21.15.64:0/3833019580 >> v1:172.21.15.120:6789/0 conn(0x7fa0380b7390 legacy=0x7fa0380b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.131+0000 7fa03daef640 1 -- 172.21.15.64:0/3833019580 >> v1:172.21.15.64:6789/0 conn(0x7fa038145bf0 legacy=0x7fa0380cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.131+0000 7fa03d2ee640 1 -- 172.21.15.64:0/3833019580 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026182090 0 0) 0x7fa0380d0d80 con 0x7fa0380b7390 2024-03-07T07:19:56.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.131+0000 7fa03daef640 1 -- 172.21.15.64:0/3833019580 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0380d1060 con 0x7fa0380d1860 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa03daef640 1 --2- 172.21.15.64:0/3833019580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0380d1860 0x7fa03806b510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0097a0 tx=0x7fa02c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bb9b7032c8fd2f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa03d2ee640 1 -- 172.21.15.64:0/3833019580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c037070 con 0x7fa0380d1860 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa03d2ee640 1 -- 172.21.15.64:0/3833019580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c030710 con 0x7fa0380d1860 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa03d2ee640 1 -- 172.21.15.64:0/3833019580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c030ae0 con 0x7fa0380d1860 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa040554640 1 -- 172.21.15.64:0/3833019580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0380d1860 msgr2=0x7fa03806b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa040554640 1 --2- 172.21.15.64:0/3833019580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0380d1860 0x7fa03806b510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0097a0 tx=0x7fa02c02b790 comp rx=0 tx=0).stop 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa040554640 1 -- 172.21.15.64:0/3833019580 shutdown_connections 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa040554640 1 --2- 172.21.15.64:0/3833019580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0380d1860 0x7fa03806b510 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa040554640 1 -- 172.21.15.64:0/3833019580 >> 172.21.15.64:0/3833019580 conn(0x7fa03806d4c0 msgr2=0x7fa038066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa040554640 1 -- 172.21.15.64:0/3833019580 shutdown_connections 2024-03-07T07:19:56.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.132+0000 7fa040554640 1 -- 172.21.15.64:0/3833019580 wait complete. 2024-03-07T07:19:56.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.133+0000 7fa040554640 1 Processor -- start 2024-03-07T07:19:56.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.133+0000 7fa040554640 1 -- start start 2024-03-07T07:19:56.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa040554640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0380b7390 0x7fa03815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa040554640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0380d1860 0x7fa03815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa040554640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa038145bf0 0x7fa03815f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa040554640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0380d0840 con 0x7fa038145bf0 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa040554640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0380d06c0 con 0x7fa0380d1860 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa040554640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03806ba50 con 0x7fa0380b7390 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa03eaf1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa038145bf0 0x7fa03815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa03eaf1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa038145bf0 0x7fa03815f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47228/0 (socket says 172.21.15.64:47228) 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa03eaf1640 1 -- 172.21.15.64:0/1510431885 learned_addr learned my addr 172.21.15.64:0/1510431885 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa03daef640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0380d1860 0x7fa03815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.134+0000 7fa03e2f0640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0380b7390 0x7fa03815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa03eaf1640 1 -- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0380b7390 msgr2=0x7fa03815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa03eaf1640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0380b7390 0x7fa03815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa03eaf1640 1 -- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0380d1860 msgr2=0x7fa03815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa03eaf1640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0380d1860 0x7fa03815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa03eaf1640 1 -- 172.21.15.64:0/1510431885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0380cc300 con 0x7fa038145bf0 2024-03-07T07:19:56.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa03eaf1640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa038145bf0 0x7fa03815f050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa02800c760 tx=0x7fa02800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa036ffd640 1 -- 172.21.15.64:0/1510431885 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028003da0 con 0x7fa038145bf0 2024-03-07T07:19:56.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa036ffd640 1 -- 172.21.15.64:0/1510431885 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028005070 con 0x7fa038145bf0 2024-03-07T07:19:56.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0380cc910 con 0x7fa038145bf0 2024-03-07T07:19:56.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa036ffd640 1 -- 172.21.15.64:0/1510431885 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028005460 con 0x7fa038145bf0 2024-03-07T07:19:56.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.135+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0380cced0 con 0x7fa038145bf0 2024-03-07T07:19:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.136+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa038062c90 con 0x7fa038145bf0 2024-03-07T07:19:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.136+0000 7fa036ffd640 1 -- 172.21.15.64:0/1510431885 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa028005600 con 0x7fa038145bf0 2024-03-07T07:19:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.137+0000 7fa036ffd640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa02406b4d0 0x7fa02406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.137+0000 7fa036ffd640 1 -- 172.21.15.64:0/1510431885 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa028089f30 con 0x7fa038145bf0 2024-03-07T07:19:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.137+0000 7fa03e2f0640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa02406b4d0 0x7fa02406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.137+0000 7fa03e2f0640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa02406b4d0 0x7fa02406d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa030000f60 tx=0x7fa030009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.141+0000 7fa036ffd640 1 -- 172.21.15.64:0/1510431885 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa0280587f0 con 0x7fa038145bf0 2024-03-07T07:19:56.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.324+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0380d26b0 con 0x7fa038145bf0 2024-03-07T07:19:56.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.325+0000 7fa036ffd640 1 -- 172.21.15.64:0/1510431885 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fa02805c890 con 0x7fa038145bf0 2024-03-07T07:19:56.326 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:56.326 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":20,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:17.288914+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa02406b4d0 msgr2=0x7fa02406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa02406b4d0 0x7fa02406d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa030000f60 tx=0x7fa030009040 comp rx=0 tx=0).stop 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa038145bf0 msgr2=0x7fa03815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa038145bf0 0x7fa03815f050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa02800c760 tx=0x7fa02800cc20 comp rx=0 tx=0).stop 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 shutdown_connections 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa02406b4d0 0x7fa02406d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa038145bf0 0x7fa03815f050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0380d1860 0x7fa03815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 --2- 172.21.15.64:0/1510431885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0380b7390 0x7fa03815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.327+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 >> 172.21.15.64:0/1510431885 conn(0x7fa03806d4c0 msgr2=0x7fa0380740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.328+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 shutdown_connections 2024-03-07T07:19:56.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.328+0000 7fa040554640 1 -- 172.21.15.64:0/1510431885 wait complete. 2024-03-07T07:19:56.339 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-07T07:19:56.339 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:56.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.444+0000 7f58c5578640 1 Processor -- start 2024-03-07T07:19:56.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.444+0000 7f58c5578640 1 -- start start 2024-03-07T07:19:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58c5578640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58c00d2c10 0x7f58c0071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58c5578640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c006bc20 0x7f58c0072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58c5578640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c00d1e70 con 0x7f58c0147270 2024-03-07T07:19:56.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58c5578640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c0147650 con 0x7f58c00d2c10 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58c5578640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00b7520 con 0x7f58c006bc20 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58b67fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58c00d2c10 0x7f58c0071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58b67fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f58c00d2c10 0x7f58c0071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47232/0 (socket says 172.21.15.64:47232) 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58b67fc640 1 -- 172.21.15.64:0/1321729714 learned_addr learned my addr 172.21.15.64:0/1321729714 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.445+0000 7f58bf7fe640 1 --2- 172.21.15.64:0/1321729714 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c006bc20 0x7f58c0072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.446+0000 7f58b67fc640 1 -- 172.21.15.64:0/1321729714 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c006bc20 msgr2=0x7f58c0072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.446+0000 7f58b67fc640 1 --2- 172.21.15.64:0/1321729714 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c006bc20 0x7f58c0072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.446+0000 7f58b67fc640 1 -- 172.21.15.64:0/1321729714 >> v1:172.21.15.120:6789/0 conn(0x7f58c0147270 legacy=0x7f58c006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.446+0000 7f58b67fc640 1 -- 172.21.15.64:0/1321729714 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00d2130 con 0x7f58c00d2c10 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.446+0000 7f58b67fc640 1 --2- 172.21.15.64:0/1321729714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c00d2c10 0x7f58c0071ee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009870 tx=0x7f58a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faef784d13cc00ca server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.446+0000 7f58be7fc640 1 -- 172.21.15.64:0/1321729714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8034070 con 0x7f58c00d2c10 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58be7fc640 1 -- 172.21.15.64:0/1321729714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a8030840 con 0x7f58c00d2c10 2024-03-07T07:19:56.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58be7fc640 1 -- 172.21.15.64:0/1321729714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8030c10 con 0x7f58c00d2c10 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 -- 172.21.15.64:0/1321729714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c00d2c10 msgr2=0x7f58c0071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 --2- 172.21.15.64:0/1321729714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c00d2c10 0x7f58c0071ee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009870 tx=0x7f58a802b980 comp rx=0 tx=0).stop 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 -- 172.21.15.64:0/1321729714 shutdown_connections 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 --2- 172.21.15.64:0/1321729714 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c006bc20 0x7f58c0072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 --2- 172.21.15.64:0/1321729714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c00d2c10 0x7f58c0071ee0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 -- 172.21.15.64:0/1321729714 >> 172.21.15.64:0/1321729714 conn(0x7f58c006d4c0 msgr2=0x7f58c0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 -- 172.21.15.64:0/1321729714 shutdown_connections 2024-03-07T07:19:56.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.447+0000 7f58c5578640 1 -- 172.21.15.64:0/1321729714 wait complete. 2024-03-07T07:19:56.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.448+0000 7f58c5578640 1 Processor -- start 2024-03-07T07:19:56.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.448+0000 7f58c5578640 1 -- start start 2024-03-07T07:19:56.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58c5578640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c006bc20 0x7f58c0140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58c5578640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c00d2c10 0x7f58c0160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58c5578640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c0147270 0x7f58c01610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58c5578640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00b71d0 con 0x7f58c0147270 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58c5578640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00d26f0 con 0x7f58c006bc20 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58c5578640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c0147810 con 0x7f58c00d2c10 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c0147270 0x7f58c01610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58bf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c0147270 0x7f58c01610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47238/0 (socket says 172.21.15.64:47238) 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58bf7fe640 1 -- 172.21.15.64:0/241552978 learned_addr learned my addr 172.21.15.64:0/241552978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58beffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c006bc20 0x7f58c0140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.449+0000 7f58b67fc640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c00d2c10 0x7f58c0160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58bf7fe640 1 -- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c00d2c10 msgr2=0x7f58c0160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58bf7fe640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c00d2c10 0x7f58c0160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58bf7fe640 1 -- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c006bc20 msgr2=0x7f58c0140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58bf7fe640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c006bc20 0x7f58c0140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58bf7fe640 1 -- 172.21.15.64:0/241552978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c01e9790 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58beffd640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c006bc20 0x7f58c0140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58bf7fe640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c0147270 0x7f58c01610c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009e30 tx=0x7f58b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58b7fff640 1 -- 172.21.15.64:0/241552978 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8018020 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58b7fff640 1 -- 172.21.15.64:0/241552978 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b8014480 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c01e8c10 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58b7fff640 1 -- 172.21.15.64:0/241552978 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8014870 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.450+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c01e91d0 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.451+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c0062c90 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.451+0000 7f58b7fff640 1 -- 172.21.15.64:0/241552978 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f58b8005d10 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.452+0000 7f58b7fff640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f58a006b4d0 0x7f58a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.452+0000 7f58b7fff640 1 -- 172.21.15.64:0/241552978 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f58b808aff0 con 0x7f58c0147270 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.452+0000 7f58beffd640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f58a006b4d0 0x7f58a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.452+0000 7f58beffd640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f58a006b4d0 0x7f58a006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58b0001ad0 tx=0x7f58b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.456+0000 7f58b7fff640 1 -- 172.21.15.64:0/241552978 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f58b8022030 con 0x7f58c0147270 2024-03-07T07:19:56.678 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.678+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c0071cc0 con 0x7f58c0147270 2024-03-07T07:19:56.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.679+0000 7f58b7fff640 1 -- 172.21.15.64:0/241552978 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7f58b8010040 con 0x7f58c0147270 2024-03-07T07:19:56.679 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:56.679 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f58a006b4d0 msgr2=0x7f58a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f58a006b4d0 0x7f58a006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58b0001ad0 tx=0x7f58b0009040 comp rx=0 tx=0).stop 2024-03-07T07:19:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c0147270 msgr2=0x7f58c01610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c0147270 0x7f58c01610c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009e30 tx=0x7f58b800c490 comp rx=0 tx=0).stop 2024-03-07T07:19:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 shutdown_connections 2024-03-07T07:19:56.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f58a006b4d0 0x7f58a006d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58c0147270 0x7f58c01610c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58c00d2c10 0x7f58c0160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 --2- 172.21.15.64:0/241552978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58c006bc20 0x7f58c0140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 >> 172.21.15.64:0/241552978 conn(0x7f58c006d4c0 msgr2=0x7f58c0162960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 shutdown_connections 2024-03-07T07:19:56.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.681+0000 7f58c5578640 1 -- 172.21.15.64:0/241552978 wait complete. 2024-03-07T07:19:56.692 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:19:56.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.793+0000 7f3f106b6640 1 Processor -- start 2024-03-07T07:19:56.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.793+0000 7f3f106b6640 1 -- start start 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.794+0000 7f3f106b6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3f0815bfd0 0x7f3f081600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.794+0000 7f3f106b6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f080b6d10 0x7f3f0815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.794+0000 7f3f106b6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f080d09f0 con 0x7f3f0806f350 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.794+0000 7f3f106b6640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f080cf4b0 con 0x7f3f080b6d10 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.794+0000 7f3f106b6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0806e7a0 con 0x7f3f0815bfd0 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0ec53640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f080b6d10 0x7f3f0815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0ec53640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3f080b6d10 0x7f3f0815c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47250/0 (socket says 172.21.15.64:47250) 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0ec53640 1 -- 172.21.15.64:0/2023107444 learned_addr learned my addr 172.21.15.64:0/2023107444 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0e452640 1 --2- 172.21.15.64:0/2023107444 >> v2:172.21.15.120:3300/0 conn(0x7f3f0815bfd0 0x7f3f081600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0ec53640 1 -- 172.21.15.64:0/2023107444 >> v2:172.21.15.120:3300/0 conn(0x7f3f0815bfd0 msgr2=0x7f3f081600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0ec53640 1 --2- 172.21.15.64:0/2023107444 >> v2:172.21.15.120:3300/0 conn(0x7f3f0815bfd0 0x7f3f081600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0ec53640 1 -- 172.21.15.64:0/2023107444 >> v1:172.21.15.120:6789/0 conn(0x7f3f0806f350 legacy=0x7f3f080b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.795+0000 7f3f0ec53640 1 -- 172.21.15.64:0/2023107444 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0806ea80 con 0x7f3f080b6d10 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f0ec53640 1 --2- 172.21.15.64:0/2023107444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f080b6d10 0x7f3f0815c3b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f08073c90 tx=0x7f3f0402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e11d253afcf469 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f0d450640 1 -- 172.21.15.64:0/2023107444 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0402cb20 con 0x7f3f080b6d10 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f0d450640 1 -- 172.21.15.64:0/2023107444 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0402ccc0 con 0x7f3f080b6d10 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f0d450640 1 -- 172.21.15.64:0/2023107444 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04034610 con 0x7f3f080b6d10 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 -- 172.21.15.64:0/2023107444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f080b6d10 msgr2=0x7f3f0815c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 --2- 172.21.15.64:0/2023107444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f080b6d10 0x7f3f0815c3b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f08073c90 tx=0x7f3f0402bf80 comp rx=0 tx=0).stop 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 -- 172.21.15.64:0/2023107444 shutdown_connections 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 --2- 172.21.15.64:0/2023107444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f080b6d10 0x7f3f0815c3b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 --2- 172.21.15.64:0/2023107444 >> v2:172.21.15.120:3300/0 conn(0x7f3f0815bfd0 0x7f3f081600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 -- 172.21.15.64:0/2023107444 >> 172.21.15.64:0/2023107444 conn(0x7f3f08071550 msgr2=0x7f3f08071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 -- 172.21.15.64:0/2023107444 shutdown_connections 2024-03-07T07:19:56.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.796+0000 7f3f106b6640 1 -- 172.21.15.64:0/2023107444 wait complete. 2024-03-07T07:19:56.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.797+0000 7f3f106b6640 1 Processor -- start 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f106b6640 1 -- start start 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f106b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3f0806f350 0x7f3f080b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f106b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f080b6d10 0x7f3f0814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f106b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0815bfd0 0x7f3f08157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f106b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806ee00 con 0x7f3f0815bfd0 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f106b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0815fbb0 con 0x7f3f080b6d10 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f106b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806ef80 con 0x7f3f0806f350 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f0ec53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0815bfd0 0x7f3f08157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f0ec53640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0815bfd0 0x7f3f08157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47252/0 (socket says 172.21.15.64:47252) 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f0ec53640 1 -- 172.21.15.64:0/3227102968 learned_addr learned my addr 172.21.15.64:0/3227102968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f0dc51640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f080b6d10 0x7f3f0814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.798+0000 7f3f0e452640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3f0806f350 0x7f3f080b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f0ec53640 1 -- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3f0806f350 msgr2=0x7f3f080b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f0ec53640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3f0806f350 0x7f3f080b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f0ec53640 1 -- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f080b6d10 msgr2=0x7f3f0814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f0ec53640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f080b6d10 0x7f3f0814cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f0ec53640 1 -- 172.21.15.64:0/3227102968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08150980 con 0x7f3f0815bfd0 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f0ec53640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0815bfd0 0x7f3f08157050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f04036040 tx=0x7f3f04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3efeffd640 1 -- 172.21.15.64:0/3227102968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04043020 con 0x7f3f0815bfd0 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3efeffd640 1 -- 172.21.15.64:0/3227102968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0403e450 con 0x7f3f0815bfd0 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f08150c60 con 0x7f3f0815bfd0 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3efeffd640 1 -- 172.21.15.64:0/3227102968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04009510 con 0x7f3f0815bfd0 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.799+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f08151220 con 0x7f3f0815bfd0 2024-03-07T07:19:56.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.800+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f08062c90 con 0x7f3f0815bfd0 2024-03-07T07:19:56.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.800+0000 7f3efeffd640 1 -- 172.21.15.64:0/3227102968 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3f04046020 con 0x7f3f0815bfd0 2024-03-07T07:19:56.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.801+0000 7f3efeffd640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3eec06b500 0x7f3eec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:56.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.801+0000 7f3efeffd640 1 -- 172.21.15.64:0/3227102968 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3f040ac7b0 con 0x7f3f0815bfd0 2024-03-07T07:19:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.801+0000 7f3f0e452640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3eec06b500 0x7f3eec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:56.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.801+0000 7f3f0e452640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3eec06b500 0x7f3eec06d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f080b5b10 tx=0x7f3f00009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:56.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:56.805+0000 7f3efeffd640 1 -- 172.21.15.64:0/3227102968 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3f0404d030 con 0x7f3f0815bfd0 2024-03-07T07:19:57.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.023+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f0815b5d0 con 0x7f3f0815bfd0 2024-03-07T07:19:57.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.024+0000 7f3efeffd640 1 -- 172.21.15.64:0/3227102968 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4977 (secure 0 0 0) 0x7f3f0407b110 con 0x7f3f0815bfd0 2024-03-07T07:19:57.025 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:57.025 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/3784554624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3784554624},{"type":"v1","addr":"172.21.15.120:6836","nonce":3784554624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/1260566534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":1260566534},{"type":"v1","addr":"172.21.15.64:6834","nonce":1260566534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1308149206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":1308149206},{"type":"v1","addr":"172.21.15.64:6835","nonce":1308149206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:19:15.653340+0000","modified":"2024-03-07T07:19:18.012811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6837/2172286760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":2172286760},{"type":"v1","addr":"172.21.15.120:6837","nonce":2172286760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4299,"qdb_cluster":[4299]},"id":1}]} 2024-03-07T07:19:57.025 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 7 2024-03-07T07:19:57.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3eec06b500 msgr2=0x7f3eec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3eec06b500 0x7f3eec06d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f080b5b10 tx=0x7f3f00009190 comp rx=0 tx=0).stop 2024-03-07T07:19:57.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0815bfd0 msgr2=0x7f3f08157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0815bfd0 0x7f3f08157050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f04036040 tx=0x7f3f04004a40 comp rx=0 tx=0).stop 2024-03-07T07:19:57.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 shutdown_connections 2024-03-07T07:19:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3eec06b500 0x7f3eec06d9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f0815bfd0 0x7f3f08157050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f080b6d10 0x7f3f0814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 --2- 172.21.15.64:0/3227102968 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3f0806f350 0x7f3f080b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 >> 172.21.15.64:0/3227102968 conn(0x7f3f08071550 msgr2=0x7f3f0814eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 shutdown_connections 2024-03-07T07:19:57.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.026+0000 7f3f106b6640 1 -- 172.21.15.64:0/3227102968 wait complete. 2024-03-07T07:19:57.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:19:57.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.137+0000 7f03737da640 1 Processor -- start 2024-03-07T07:19:57.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.138+0000 7f03737da640 1 -- start start 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.138+0000 7f03737da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.138+0000 7f03737da640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f036c072cf0 0x7f036c071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.138+0000 7f03737da640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c071c50 con 0x7f036c0d0990 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.138+0000 7f03737da640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c06c3a0 con 0x7f036c072cf0 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.138+0000 7f03737da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c071f30 con 0x7f036c15f170 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0371576640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0370d75640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f036c072cf0 0x7f036c071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0371d77640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f036c0d0990 0x7f036c1470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:37564/0 (socket says 172.21.15.64:37564) 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0371d77640 1 -- 172.21.15.64:0/1940298619 learned_addr learned my addr 172.21.15.64:0/1940298619 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:57.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0371576640 1 -- 172.21.15.64:0/1940298619 >> v2:172.21.15.120:3300/0 conn(0x7f036c072cf0 msgr2=0x7f036c071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0371576640 1 --2- 172.21.15.64:0/1940298619 >> v2:172.21.15.120:3300/0 conn(0x7f036c072cf0 0x7f036c071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0371576640 1 -- 172.21.15.64:0/1940298619 >> v1:172.21.15.120:6789/0 conn(0x7f036c0d0990 legacy=0x7f036c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.139+0000 7f0371576640 1 -- 172.21.15.64:0/1940298619 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c072210 con 0x7f036c15f170 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f0371576640 1 --2- 172.21.15.64:0/1940298619 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c15e650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03680097a0 tx=0x7f036802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a214baa8f584e1a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f0363fff640 1 -- 172.21.15.64:0/1940298619 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368035020 con 0x7f036c15f170 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f0363fff640 1 -- 172.21.15.64:0/1940298619 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0368031830 con 0x7f036c15f170 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f0363fff640 1 -- 172.21.15.64:0/1940298619 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368031c00 con 0x7f036c15f170 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 -- 172.21.15.64:0/1940298619 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 msgr2=0x7f036c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 --2- 172.21.15.64:0/1940298619 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c15e650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03680097a0 tx=0x7f036802b8d0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 -- 172.21.15.64:0/1940298619 shutdown_connections 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 --2- 172.21.15.64:0/1940298619 >> v2:172.21.15.120:3300/0 conn(0x7f036c072cf0 0x7f036c071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 --2- 172.21.15.64:0/1940298619 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c15e650 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 -- 172.21.15.64:0/1940298619 >> 172.21.15.64:0/1940298619 conn(0x7f036c06b480 msgr2=0x7f036c06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:57.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 -- 172.21.15.64:0/1940298619 shutdown_connections 2024-03-07T07:19:57.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.140+0000 7f03737da640 1 -- 172.21.15.64:0/1940298619 wait complete. 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 Processor -- start 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 -- start start 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f036c072cf0 0x7f036c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f036c0d0990 0x7f036c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0d0dd0 con 0x7f036c072cf0 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0727d0 con 0x7f036c0d0990 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f03737da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0d0f50 con 0x7f036c15f170 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f0371576640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f036c072cf0 0x7f036c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.142+0000 7f0371576640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f036c072cf0 0x7f036c157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47264/0 (socket says 172.21.15.64:47264) 2024-03-07T07:19:57.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371576640 1 -- 172.21.15.64:0/1782396806 learned_addr learned my addr 172.21.15.64:0/1782396806 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0370d75640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f036c0d0990 0x7f036c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371d77640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371576640 1 -- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 msgr2=0x7f036c0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371576640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c0b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371576640 1 -- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f036c0d0990 msgr2=0x7f036c159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371576640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f036c0d0990 0x7f036c159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371576640 1 -- 172.21.15.64:0/1782396806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c158080 con 0x7f036c072cf0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0371576640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f036c072cf0 0x7f036c157b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0368000c00 tx=0x7f03680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0361ffb640 1 -- 172.21.15.64:0/1782396806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368040070 con 0x7f036c072cf0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0361ffb640 1 -- 172.21.15.64:0/1782396806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036803a850 con 0x7f036c072cf0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f0361ffb640 1 -- 172.21.15.64:0/1782396806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368035040 con 0x7f036c072cf0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c0b3920 con 0x7f036c072cf0 2024-03-07T07:19:57.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.143+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c159090 con 0x7f036c072cf0 2024-03-07T07:19:57.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.144+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c062c90 con 0x7f036c072cf0 2024-03-07T07:19:57.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.145+0000 7f0361ffb640 1 -- 172.21.15.64:0/1782396806 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f036804b060 con 0x7f036c072cf0 2024-03-07T07:19:57.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.145+0000 7f0361ffb640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f035006b4d0 0x7f035006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.145+0000 7f0361ffb640 1 -- 172.21.15.64:0/1782396806 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f03680aba60 con 0x7f036c072cf0 2024-03-07T07:19:57.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.145+0000 7f0370d75640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f035006b4d0 0x7f035006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.146+0000 7f0370d75640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f035006b4d0 0x7f035006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f036c0cf0d0 tx=0x7f035c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.150+0000 7f0361ffb640 1 -- 172.21.15.64:0/1782396806 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f036c158080 con 0x7f036c072cf0 2024-03-07T07:19:57.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.331+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f036c0718d0 con 0x7f036c072cf0 2024-03-07T07:19:57.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.332+0000 7f0361ffb640 1 -- 172.21.15.64:0/1782396806 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16819 (secure 0 0 0) 0x7f0368031800 con 0x7f036c072cf0 2024-03-07T07:19:57.332 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:19:57.332 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:19:56.419127+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:19:14.236451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T07:19:14.610190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f035006b4d0 msgr2=0x7f035006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f035006b4d0 0x7f035006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f036c0cf0d0 tx=0x7f035c009190 comp rx=0 tx=0).stop 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f036c072cf0 msgr2=0x7f036c157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f036c072cf0 0x7f036c157b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0368000c00 tx=0x7f03680046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 shutdown_connections 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f035006b4d0 0x7f035006d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f036c15f170 0x7f036c0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f036c0d0990 0x7f036c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 --2- 172.21.15.64:0/1782396806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f036c072cf0 0x7f036c157b40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 >> 172.21.15.64:0/1782396806 conn(0x7f036c06b480 msgr2=0x7f036c0b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:57.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 shutdown_connections 2024-03-07T07:19:57.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.334+0000 7f03737da640 1 -- 172.21.15.64:0/1782396806 wait complete. 2024-03-07T07:19:57.345 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-07T07:19:57.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.451+0000 7f10d5314640 1 Processor -- start 2024-03-07T07:19:57.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.451+0000 7f10d5314640 1 -- start start 2024-03-07T07:19:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.451+0000 7f10d5314640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f10d0162030 0x7f10d015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.451+0000 7f10d5314640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10d01416d0 0x7f10d0162410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10d5314640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d0141c60 con 0x7f10d0162950 2024-03-07T07:19:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10d5314640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d00b44b0 con 0x7f10d01416d0 2024-03-07T07:19:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10d5314640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d015ecd0 con 0x7f10d0162030 2024-03-07T07:19:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ce7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10d01416d0 0x7f10d0162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ce7fc640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f10d01416d0 0x7f10d0162410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47270/0 (socket says 172.21.15.64:47270) 2024-03-07T07:19:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ce7fc640 1 -- 172.21.15.64:0/1606430597 learned_addr learned my addr 172.21.15.64:0/1606430597 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ceffd640 1 --2- 172.21.15.64:0/1606430597 >> v2:172.21.15.120:3300/0 conn(0x7f10d0162030 0x7f10d015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ce7fc640 1 -- 172.21.15.64:0/1606430597 >> v2:172.21.15.120:3300/0 conn(0x7f10d0162030 msgr2=0x7f10d015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ce7fc640 1 --2- 172.21.15.64:0/1606430597 >> v2:172.21.15.120:3300/0 conn(0x7f10d0162030 0x7f10d015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ce7fc640 1 -- 172.21.15.64:0/1606430597 >> v1:172.21.15.120:6789/0 conn(0x7f10d0162950 legacy=0x7f10d00d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.452+0000 7f10ce7fc640 1 -- 172.21.15.64:0/1606430597 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d015efb0 con 0x7f10d01416d0 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10ce7fc640 1 --2- 172.21.15.64:0/1606430597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d0162410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f10d0074d40 tx=0x7f10c402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ef2986ed63c8e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10cdffb640 1 -- 172.21.15.64:0/1606430597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4034070 con 0x7f10d01416d0 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10cdffb640 1 -- 172.21.15.64:0/1606430597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c4030860 con 0x7f10d01416d0 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10cdffb640 1 -- 172.21.15.64:0/1606430597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4030c30 con 0x7f10d01416d0 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 -- 172.21.15.64:0/1606430597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 msgr2=0x7f10d0162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 --2- 172.21.15.64:0/1606430597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d0162410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f10d0074d40 tx=0x7f10c402b9a0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 -- 172.21.15.64:0/1606430597 shutdown_connections 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 --2- 172.21.15.64:0/1606430597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d0162410 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 --2- 172.21.15.64:0/1606430597 >> v2:172.21.15.120:3300/0 conn(0x7f10d0162030 0x7f10d015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 -- 172.21.15.64:0/1606430597 >> 172.21.15.64:0/1606430597 conn(0x7f10d0070400 msgr2=0x7f10d00d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 -- 172.21.15.64:0/1606430597 shutdown_connections 2024-03-07T07:19:57.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.453+0000 7f10d5314640 1 -- 172.21.15.64:0/1606430597 wait complete. 2024-03-07T07:19:57.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 Processor -- start 2024-03-07T07:19:57.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 -- start start 2024-03-07T07:19:57.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d00d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10d0162030 0x7f10d01402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10d0162950 0x7f10d0159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00d3ba0 con 0x7f10d01416d0 2024-03-07T07:19:57.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d015f570 con 0x7f10d0162950 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10d5314640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00d3d20 con 0x7f10d0162030 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.455+0000 7f10ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d00d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d00d2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47272/0 (socket says 172.21.15.64:47272) 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 -- 172.21.15.64:0/2832389064 learned_addr learned my addr 172.21.15.64:0/2832389064 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10cf7fe640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10d0162950 0x7f10d0159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ce7fc640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10d0162030 0x7f10d01402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 -- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10d0162030 msgr2=0x7f10d01402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10d0162030 0x7f10d01402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 -- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10d0162950 msgr2=0x7f10d0159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10d0162950 0x7f10d0159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 -- 172.21.15.64:0/2832389064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0159c60 con 0x7f10d01416d0 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10ceffd640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d00d2ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e9b0 tx=0x7f10c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10b77fe640 1 -- 172.21.15.64:0/2832389064 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0017070 con 0x7f10d01416d0 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10b77fe640 1 -- 172.21.15.64:0/2832389064 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c000c9a0 con 0x7f10d01416d0 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d00d2120 con 0x7f10d01416d0 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10b77fe640 1 -- 172.21.15.64:0/2832389064 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000cd90 con 0x7f10d01416d0 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.456+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d00d26e0 con 0x7f10d01416d0 2024-03-07T07:19:57.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.457+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d0062c90 con 0x7f10d01416d0 2024-03-07T07:19:57.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.458+0000 7f10b77fe640 1 -- 172.21.15.64:0/2832389064 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f10c001a070 con 0x7f10d01416d0 2024-03-07T07:19:57.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.458+0000 7f10b77fe640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f10bc06b4d0 0x7f10bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.458+0000 7f10b77fe640 1 -- 172.21.15.64:0/2832389064 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f10c008a1e0 con 0x7f10d01416d0 2024-03-07T07:19:57.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.458+0000 7f10ce7fc640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f10bc06b4d0 0x7f10bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.459+0000 7f10ce7fc640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f10bc06b4d0 0x7f10bc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f10c4036000 tx=0x7f10c403b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.462+0000 7f10b77fe640 1 -- 172.21.15.64:0/2832389064 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f10c0058990 con 0x7f10d01416d0 2024-03-07T07:19:57.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.683+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f10d0143140 con 0x7f10d01416d0 2024-03-07T07:19:57.683 INFO:tasks.ceph.mon.a.smithi064.stderr:2024-03-07T07:19:57.683+0000 7f43fd28a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T07:19:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.684+0000 7f10b77fe640 1 -- 172.21.15.64:0/2832389064 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f10c005ca30 con 0x7f10d01416d0 2024-03-07T07:19:57.684 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-07T07:19:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f10bc06b4d0 msgr2=0x7f10bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f10bc06b4d0 0x7f10bc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f10c4036000 tx=0x7f10c403b040 comp rx=0 tx=0).stop 2024-03-07T07:19:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 msgr2=0x7f10d00d2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d00d2ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e9b0 tx=0x7f10c000ee70 comp rx=0 tx=0).stop 2024-03-07T07:19:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 shutdown_connections 2024-03-07T07:19:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f10bc06b4d0 0x7f10bc06d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10d0162950 0x7f10d0159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10d0162030 0x7f10d01402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 --2- 172.21.15.64:0/2832389064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10d01416d0 0x7f10d00d2ee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 >> 172.21.15.64:0/2832389064 conn(0x7f10d0070400 msgr2=0x7f10d0160fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 shutdown_connections 2024-03-07T07:19:57.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.687+0000 7f10d5314640 1 -- 172.21.15.64:0/2832389064 wait complete. 2024-03-07T07:19:57.689 INFO:tasks.ceph.mds.b.smithi120.stderr: -8163> 2024-03-07T07:19:09.742+0000 7fd3e99fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-07T07:19:57.690 INFO:tasks.ceph.mds.b.smithi120.stderr: -7021> 2024-03-07T07:19:16.755+0000 7fd3e39f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T07:19:57.698 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-07T07:19:57.721 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-03-07T07:19:57.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.801+0000 7fc6d5dc9640 1 Processor -- start 2024-03-07T07:19:57.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6d5dc9640 1 -- start start 2024-03-07T07:19:57.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6d5dc9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc6d015e1d0 0x7fc6d0162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6d5dc9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d0070e10 0x7fc6d0161e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6d5dc9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d00715b0 con 0x7fc6d01623b0 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6d5dc9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d00d2220 con 0x7fc6d015e1d0 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6d5dc9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d00d2500 con 0x7fc6d0070e10 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6cf7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc6d015e1d0 0x7fc6d0162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6cffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc6d01623b0 0x7fc6d00ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45398/0 (socket says 172.21.15.64:45398) 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.802+0000 7fc6cffff640 1 -- 172.21.15.64:0/2376799853 learned_addr learned my addr 172.21.15.64:0/2376799853 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6ceffd640 1 --2- 172.21.15.64:0/2376799853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d0070e10 0x7fc6d0161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6cf7fe640 1 -- 172.21.15.64:0/2376799853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d0070e10 msgr2=0x7fc6d0161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6cf7fe640 1 --2- 172.21.15.64:0/2376799853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d0070e10 0x7fc6d0161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6cf7fe640 1 -- 172.21.15.64:0/2376799853 >> v1:172.21.15.64:6789/0 conn(0x7fc6d01623b0 legacy=0x7fc6d00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6cf7fe640 1 -- 172.21.15.64:0/2376799853 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d0071890 con 0x7fc6d015e1d0 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6ce7fc640 1 -- 172.21.15.64:0/2376799853 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119791030 0 0) 0x7fc6d00715b0 con 0x7fc6d01623b0 2024-03-07T07:19:57.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6ceffd640 1 --2- 172.21.15.64:0/2376799853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d0070e10 0x7fc6d0161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:19:57.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.803+0000 7fc6cf7fe640 1 --2- 172.21.15.64:0/2376799853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d015e1d0 0x7fc6d0162a10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00d23a0 tx=0x7fc6c002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07f7e89029e14f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6ce7fc640 1 -- 172.21.15.64:0/2376799853 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0031770 con 0x7fc6d015e1d0 2024-03-07T07:19:57.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6ce7fc640 1 -- 172.21.15.64:0/2376799853 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c0037070 con 0x7fc6d015e1d0 2024-03-07T07:19:57.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6ce7fc640 1 -- 172.21.15.64:0/2376799853 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0031b30 con 0x7fc6d015e1d0 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/2376799853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d015e1d0 msgr2=0x7fc6d0162a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/2376799853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d015e1d0 0x7fc6d0162a10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00d23a0 tx=0x7fc6c002b810 comp rx=0 tx=0).stop 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/2376799853 shutdown_connections 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/2376799853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d0070e10 0x7fc6d0161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/2376799853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d015e1d0 0x7fc6d0162a10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/2376799853 >> 172.21.15.64:0/2376799853 conn(0x7fc6d00d4230 msgr2=0x7fc6d00d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/2376799853 shutdown_connections 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.804+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/2376799853 wait complete. 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.805+0000 7fc6d5dc9640 1 Processor -- start 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6d5dc9640 1 -- start start 2024-03-07T07:19:57.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6d5dc9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d0070e10 0x7fc6d00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6d5dc9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d015e1d0 0x7fc6d00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6d5dc9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6d01623b0 0x7fc6d014a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6d5dc9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d0071270 con 0x7fc6d0070e10 2024-03-07T07:19:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6d5dc9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d01627e0 con 0x7fc6d01623b0 2024-03-07T07:19:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6d5dc9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d015e5b0 con 0x7fc6d015e1d0 2024-03-07T07:19:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6cf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d0070e10 0x7fc6d00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6cf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d0070e10 0x7fc6d00cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47280/0 (socket says 172.21.15.64:47280) 2024-03-07T07:19:57.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.806+0000 7fc6cf7fe640 1 -- 172.21.15.64:0/3659355715 learned_addr learned my addr 172.21.15.64:0/3659355715 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6ceffd640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d015e1d0 0x7fc6d00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cffff640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6d01623b0 0x7fc6d014a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cf7fe640 1 -- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d015e1d0 msgr2=0x7fc6d00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cf7fe640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d015e1d0 0x7fc6d00ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cf7fe640 1 -- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6d01623b0 msgr2=0x7fc6d014a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cf7fe640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6d01623b0 0x7fc6d014a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cf7fe640 1 -- 172.21.15.64:0/3659355715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d015fa30 con 0x7fc6d0070e10 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6ceffd640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d015e1d0 0x7fc6d00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cffff640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6d01623b0 0x7fc6d014a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6cf7fe640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d0070e10 0x7fc6d00cd910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000c00 tx=0x7fc6c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6b7fff640 1 -- 172.21.15.64:0/3659355715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0032810 con 0x7fc6d0070e10 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6b7fff640 1 -- 172.21.15.64:0/3659355715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c00329d0 con 0x7fc6d0070e10 2024-03-07T07:19:57.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d015fd30 con 0x7fc6d0070e10 2024-03-07T07:19:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6b7fff640 1 -- 172.21.15.64:0/3659355715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0032dc0 con 0x7fc6d0070e10 2024-03-07T07:19:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.807+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d0160570 con 0x7fc6d0070e10 2024-03-07T07:19:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.808+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d0062c90 con 0x7fc6d0070e10 2024-03-07T07:19:57.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.808+0000 7fc6b7fff640 1 -- 172.21.15.64:0/3659355715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc6c004c060 con 0x7fc6d0070e10 2024-03-07T07:19:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.809+0000 7fc6b7fff640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc6bc06b4d0 0x7fc6bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.809+0000 7fc6b7fff640 1 -- 172.21.15.64:0/3659355715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fc6c00abc00 con 0x7fc6d0070e10 2024-03-07T07:19:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.809+0000 7fc6ceffd640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc6bc06b4d0 0x7fc6bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.810+0000 7fc6ceffd640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc6bc06b4d0 0x7fc6bc06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:57.813+0000 7fc6b7fff640 1 -- 172.21.15.64:0/3659355715 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc6c00b2080 con 0x7fc6d0070e10 2024-03-07T07:19:58.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.034+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc6d00d1dd0 con 0x7fc6d0070e10 2024-03-07T07:19:58.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.690+0000 7fc6b7fff640 1 -- 172.21.15.64:0/3659355715 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc6d015fd30 con 0x7fc6d0070e10 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc6bc06b4d0 msgr2=0x7fc6bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc6bc06b4d0 0x7fc6bc06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4008040 comp rx=0 tx=0).stop 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d0070e10 msgr2=0x7fc6d00cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d0070e10 0x7fc6d00cd910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000c00 tx=0x7fc6c00046c0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 shutdown_connections 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc6bc06b4d0 0x7fc6bc06d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6d01623b0 0x7fc6d014a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6d015e1d0 0x7fc6d00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 --2- 172.21.15.64:0/3659355715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6d0070e10 0x7fc6d00cd910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 >> 172.21.15.64:0/3659355715 conn(0x7fc6d00d4230 msgr2=0x7fc6d00b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 shutdown_connections 2024-03-07T07:19:58.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.693+0000 7fc6d5dc9640 1 -- 172.21.15.64:0/3659355715 wait complete. 2024-03-07T07:19:58.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-07T07:19:58.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.811+0000 7ff71f9da640 1 Processor -- start 2024-03-07T07:19:58.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.812+0000 7ff71f9da640 1 -- start start 2024-03-07T07:19:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.812+0000 7ff71f9da640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff7180b2e00 0x7ff718162750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.812+0000 7ff71f9da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b6600 0x7ff7180b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.812+0000 7ff71f9da640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff718162c90 con 0x7ff7180b31e0 2024-03-07T07:19:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.812+0000 7ff71f9da640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71815e860 con 0x7ff7180b2e00 2024-03-07T07:19:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.812+0000 7ff71f9da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71815eb40 con 0x7ff7180b6600 2024-03-07T07:19:58.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71df77640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b6600 0x7ff7180b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71cf75640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7ff7180b31e0 0x7ff7180b5ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:37582/0 (socket says 172.21.15.64:37582) 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71cf75640 1 -- 172.21.15.64:0/2279557965 learned_addr learned my addr 172.21.15.64:0/2279557965 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71d776640 1 --2- 172.21.15.64:0/2279557965 >> v2:172.21.15.120:3300/0 conn(0x7ff7180b2e00 0x7ff718162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff70ffff640 1 -- 172.21.15.64:0/2279557965 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841576485 0 0) 0x7ff708001110 con 0x7ff7180b31e0 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff70ffff640 1 -- 172.21.15.64:0/2279557965 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff704003520 con 0x7ff7180b31e0 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71d776640 1 -- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b6600 msgr2=0x7ff7180b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71d776640 1 --2- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b6600 0x7ff7180b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71d776640 1 -- 172.21.15.64:0/2279557965 >> v1:172.21.15.120:6789/0 conn(0x7ff7180b31e0 legacy=0x7ff7180b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71d776640 1 -- 172.21.15.64:0/2279557965 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71815ee20 con 0x7ff7180b2e00 2024-03-07T07:19:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.813+0000 7ff71df77640 1 --2- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b6600 0x7ff7180b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.814+0000 7ff71d776640 1 --2- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b2e00 0x7ff718162750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff7140097a0 tx=0x7ff71402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5088962dac560adb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.814+0000 7ff70ffff640 1 -- 172.21.15.64:0/2279557965 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7140315a0 con 0x7ff7180b2e00 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.814+0000 7ff70ffff640 1 -- 172.21.15.64:0/2279557965 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714037070 con 0x7ff7180b2e00 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.814+0000 7ff70ffff640 1 -- 172.21.15.64:0/2279557965 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714031960 con 0x7ff7180b2e00 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 -- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b2e00 msgr2=0x7ff718162750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b2e00 0x7ff718162750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff7140097a0 tx=0x7ff71402b840 comp rx=0 tx=0).stop 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 -- 172.21.15.64:0/2279557965 shutdown_connections 2024-03-07T07:19:58.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b6600 0x7ff7180b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2279557965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b2e00 0x7ff718162750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 -- 172.21.15.64:0/2279557965 >> 172.21.15.64:0/2279557965 conn(0x7ff7180d0f50 msgr2=0x7ff7180d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:58.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 -- 172.21.15.64:0/2279557965 shutdown_connections 2024-03-07T07:19:58.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.815+0000 7ff71f9da640 1 -- 172.21.15.64:0/2279557965 wait complete. 2024-03-07T07:19:58.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.816+0000 7ff71f9da640 1 Processor -- start 2024-03-07T07:19:58.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.816+0000 7ff71f9da640 1 -- start start 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71f9da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b2e00 0x7ff7180c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71f9da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7180b31e0 0x7ff7181568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71f9da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b6600 0x7ff718158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71f9da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180cf270 con 0x7ff7180b31e0 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71f9da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180cf0f0 con 0x7ff7180b6600 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71f9da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180cf3f0 con 0x7ff7180b2e00 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7180b31e0 0x7ff7181568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7180b31e0 0x7ff7181568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47282/0 (socket says 172.21.15.64:47282) 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 -- 172.21.15.64:0/2876046065 learned_addr learned my addr 172.21.15.64:0/2876046065 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:58.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71d776640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b2e00 0x7ff7180c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71df77640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b6600 0x7ff718158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 -- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b2e00 msgr2=0x7ff7180c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b2e00 0x7ff7180c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 -- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b6600 msgr2=0x7ff718158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b6600 0x7ff718158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 -- 172.21.15.64:0/2876046065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7181571c0 con 0x7ff7180b31e0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.817+0000 7ff71cf75640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7180b31e0 0x7ff7181568c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c400 tx=0x7ff70800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.818+0000 7ff70dffb640 1 -- 172.21.15.64:0/2876046065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708017070 con 0x7ff7180b31e0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.818+0000 7ff70dffb640 1 -- 172.21.15.64:0/2876046065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708005ce0 con 0x7ff7180b31e0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.818+0000 7ff70dffb640 1 -- 172.21.15.64:0/2876046065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080072d0 con 0x7ff7180b31e0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.818+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7181f7390 con 0x7ff7180b31e0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.818+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff718156e00 con 0x7ff7180b31e0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.819+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff718062c90 con 0x7ff7180b31e0 2024-03-07T07:19:58.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.819+0000 7ff70dffb640 1 -- 172.21.15.64:0/2876046065 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff70801a070 con 0x7ff7180b31e0 2024-03-07T07:19:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.820+0000 7ff70dffb640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.820+0000 7ff70dffb640 1 -- 172.21.15.64:0/2876046065 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7ff708089500 con 0x7ff7180b31e0 2024-03-07T07:19:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.820+0000 7ff71d776640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:58.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.820+0000 7ff71d776640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff714036000 tx=0x7ff71403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:58.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:58.824+0000 7ff70dffb640 1 -- 172.21.15.64:0/2876046065 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff718162c90 con 0x7ff7180b31e0 2024-03-07T07:19:59.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.005+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff718154e80 con 0x7ff7180b31e0 2024-03-07T07:19:59.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.691+0000 7ff70dffb640 1 -- 172.21.15.64:0/2876046065 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff708057cf0 con 0x7ff7180b31e0 2024-03-07T07:19:59.691 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.694+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff6fc06b4d0 msgr2=0x7ff6fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.694+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff714036000 tx=0x7ff71403d040 comp rx=0 tx=0).stop 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.694+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7180b31e0 msgr2=0x7ff7181568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.694+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7180b31e0 0x7ff7181568c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c400 tx=0x7ff70800c8c0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 shutdown_connections 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff7180b31e0 0x7ff7181568c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7180b6600 0x7ff718158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 --2- 172.21.15.64:0/2876046065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7180b2e00 0x7ff7180c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 >> 172.21.15.64:0/2876046065 conn(0x7ff7180d0f50 msgr2=0x7ff71815cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 shutdown_connections 2024-03-07T07:19:59.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.695+0000 7ff71f9da640 1 -- 172.21.15.64:0/2876046065 wait complete. 2024-03-07T07:19:59.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-07T07:19:59.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.820+0000 7f053cadd640 1 Processor -- start 2024-03-07T07:19:59.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.820+0000 7f053cadd640 1 -- start start 2024-03-07T07:19:59.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.821+0000 7f053cadd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05380d21b0 0x7f05380b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.821+0000 7f053cadd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05380d2670 con 0x7f053815e010 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.821+0000 7f053cadd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05380d2950 con 0x7f05380b7910 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.821+0000 7f053cadd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05380d2c30 con 0x7f05380d21b0 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.821+0000 7f0535d9b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05380d21b0 0x7f05380b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.821+0000 7f053659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f053815e010 0x7f05380d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:45402/0 (socket says 172.21.15.64:45402) 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.821+0000 7f053659c640 1 -- 172.21.15.64:0/1475938630 learned_addr learned my addr 172.21.15.64:0/1475938630 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.822+0000 7f0535d9b640 1 -- 172.21.15.64:0/1475938630 >> v1:172.21.15.120:6789/0 conn(0x7f05380b7910 legacy=0x7f053815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.822+0000 7f0535d9b640 1 -- 172.21.15.64:0/1475938630 >> v1:172.21.15.64:6789/0 conn(0x7f053815e010 legacy=0x7f05380d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.822+0000 7f053559a640 1 -- 172.21.15.64:0/1475938630 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558781899 0 0) 0x7f05380d2670 con 0x7f053815e010 2024-03-07T07:19:59.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.822+0000 7f0535d9b640 1 -- 172.21.15.64:0/1475938630 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380d2f10 con 0x7f05380d21b0 2024-03-07T07:19:59.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.822+0000 7f0535d9b640 1 --2- 172.21.15.64:0/1475938630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f05380b64a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f052c0097a0 tx=0x7f052c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71af8c3db183bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053559a640 1 -- 172.21.15.64:0/1475938630 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0315a0 con 0x7f05380d21b0 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053559a640 1 -- 172.21.15.64:0/1475938630 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c037070 con 0x7f05380d21b0 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053559a640 1 -- 172.21.15.64:0/1475938630 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c031960 con 0x7f05380d21b0 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053cadd640 1 -- 172.21.15.64:0/1475938630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 msgr2=0x7f05380b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053cadd640 1 --2- 172.21.15.64:0/1475938630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f05380b64a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f052c0097a0 tx=0x7f052c02b840 comp rx=0 tx=0).stop 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053cadd640 1 -- 172.21.15.64:0/1475938630 shutdown_connections 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053cadd640 1 --2- 172.21.15.64:0/1475938630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f05380b64a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053cadd640 1 -- 172.21.15.64:0/1475938630 >> 172.21.15.64:0/1475938630 conn(0x7f0538072c10 msgr2=0x7f0538073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053cadd640 1 -- 172.21.15.64:0/1475938630 shutdown_connections 2024-03-07T07:19:59.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.823+0000 7f053cadd640 1 -- 172.21.15.64:0/1475938630 wait complete. 2024-03-07T07:19:59.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 Processor -- start 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 -- start start 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05380b7910 0x7f0538144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f053815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053815e010 0x7f05381596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05380b6b60 con 0x7f05380d21b0 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05380b69e0 con 0x7f053815e010 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f053cadd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05380b7dc0 con 0x7f05380b7910 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.825+0000 7f0535d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f053815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f053815c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47304/0 (socket says 172.21.15.64:47304) 2024-03-07T07:19:59.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 -- 172.21.15.64:0/3232376751 learned_addr learned my addr 172.21.15.64:0/3232376751 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0536d9d640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053815e010 0x7f05381596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f053659c640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05380b7910 0x7f0538144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 -- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05380b7910 msgr2=0x7f0538144e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05380b7910 0x7f0538144e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 -- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053815e010 msgr2=0x7f05381596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053815e010 0x7f05381596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 -- 172.21.15.64:0/3232376751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05381453c0 con 0x7f05380d21b0 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0536d9d640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053815e010 0x7f05381596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f053659c640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05380b7910 0x7f0538144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0535d9b640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f053815c730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f052c0097a0 tx=0x7f052c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0526ffd640 1 -- 172.21.15.64:0/3232376751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c032840 con 0x7f05380d21b0 2024-03-07T07:19:59.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0526ffd640 1 -- 172.21.15.64:0/3232376751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c0329e0 con 0x7f05380d21b0 2024-03-07T07:19:59.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053815b7a0 con 0x7f05380d21b0 2024-03-07T07:19:59.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f0526ffd640 1 -- 172.21.15.64:0/3232376751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c032dd0 con 0x7f05380d21b0 2024-03-07T07:19:59.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.826+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053815bd60 con 0x7f05380d21b0 2024-03-07T07:19:59.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.827+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0538062c90 con 0x7f05380d21b0 2024-03-07T07:19:59.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.828+0000 7f0526ffd640 1 -- 172.21.15.64:0/3232376751 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f052c03b3f0 con 0x7f05380d21b0 2024-03-07T07:19:59.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.828+0000 7f0526ffd640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f051806b4d0 0x7f051806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:19:59.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.828+0000 7f0526ffd640 1 -- 172.21.15.64:0/3232376751 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f052c02f030 con 0x7f05380d21b0 2024-03-07T07:19:59.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.828+0000 7f053659c640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f051806b4d0 0x7f051806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:19:59.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.829+0000 7f053659c640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f051806b4d0 0x7f051806d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0530000f60 tx=0x7f0530009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:19:59.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:19:59.832+0000 7f0526ffd640 1 -- 172.21.15.64:0/3232376751 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f052c040070 con 0x7f05380d21b0 2024-03-07T07:20:00.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.018+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0538162850 con 0x7f05380d21b0 2024-03-07T07:20:00.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.696+0000 7f0526ffd640 1 -- 172.21.15.64:0/3232376751 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f052c07ae80 con 0x7f05380d21b0 2024-03-07T07:20:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-03-07T07:20:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f051806b4d0 msgr2=0x7f051806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f051806b4d0 0x7f051806d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0530000f60 tx=0x7f0530009040 comp rx=0 tx=0).stop 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 msgr2=0x7f053815c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f053815c730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f052c0097a0 tx=0x7f052c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 shutdown_connections 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f051806b4d0 0x7f051806d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05380d21b0 0x7f053815c730 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053815e010 0x7f05381596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 --2- 172.21.15.64:0/3232376751 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05380b7910 0x7f0538144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 >> 172.21.15.64:0/3232376751 conn(0x7f0538072c10 msgr2=0x7f0538070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.701+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 shutdown_connections 2024-03-07T07:20:00.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.702+0000 7f053cadd640 1 -- 172.21.15.64:0/3232376751 wait complete. 2024-03-07T07:20:00.716 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-07T07:20:00.716 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-07T07:20:00.716 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-07T07:20:00.716 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:20:00.717 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-07T07:20:00.717 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-07T07:20:00.717 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-07T07:20:00.717 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:20:00.717 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-07T07:20:00.717 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-07T07:20:00.717 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-07T07:20:00.717 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:20:00.718 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-07T07:20:00.718 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-07T07:20:00.718 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-07T07:20:00.718 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:20:00.726 INFO:tasks.ceph.mds.a:Stopped 2024-03-07T07:20:00.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-07T07:20:00.727 INFO:tasks.ceph.mds.c:Stopped 2024-03-07T07:20:00.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-07T07:20:00.731 INFO:tasks.ceph.mds.d:Stopped 2024-03-07T07:20:00.731 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-07T07:20:00.732 INFO:tasks.ceph.mds.b:Stopped 2024-03-07T07:20:00.732 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-07T07:20:00.769 INFO:tasks.ceph.mds.a:Started 2024-03-07T07:20:00.769 DEBUG:teuthology.parallel:result is None 2024-03-07T07:20:00.770 INFO:tasks.ceph.mds.c:Started 2024-03-07T07:20:00.770 DEBUG:teuthology.parallel:result is None 2024-03-07T07:20:00.774 INFO:tasks.ceph.mds.d:Started 2024-03-07T07:20:00.775 DEBUG:teuthology.parallel:result is None 2024-03-07T07:20:00.775 INFO:tasks.ceph.mds.b:Started 2024-03-07T07:20:00.775 DEBUG:teuthology.parallel:result is None 2024-03-07T07:20:00.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-07T07:20:00.849 INFO:tasks.ceph.mds.a.smithi064.stdout:starting mds.a at 2024-03-07T07:20:00.850 INFO:tasks.ceph.mds.c.smithi064.stdout:starting mds.c at 2024-03-07T07:20:00.850 INFO:tasks.ceph.mds.c.smithi064.stderr:2024-03-07T07:20:00.850+0000 7f656e5fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-03-07T07:20:00.857 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-03-07T07:20:00.857 INFO:tasks.ceph.mds.d.smithi120.stdout:starting mds.d at 2024-03-07T07:20:00.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.891+0000 7f4603a94640 1 Processor -- start 2024-03-07T07:20:00.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.891+0000 7f4603a94640 1 -- start start 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4603a94640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f45fc140ec0 0x7f45fc1416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4603a94640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4603a94640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45fc15e6a0 0x7f45fc15eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4603a94640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45fc15f420 con 0x7f45fc15e6a0 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4603a94640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45fc0b4590 con 0x7f45fc140ec0 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4603a94640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc0b4870 con 0x7f45fc161c90 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45fc15e6a0 0x7f45fc15eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f45fc15e6a0 0x7f45fc15eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35182/0 (socket says 172.21.15.64:35182) 2024-03-07T07:20:00.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 -- 172.21.15.64:0/1079236039 learned_addr learned my addr 172.21.15.64:0/1079236039 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4601830640 1 --2- 172.21.15.64:0/1079236039 >> v2:172.21.15.120:3300/0 conn(0x7f45fc140ec0 0x7f45fc1416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f460102f640 1 --2- 172.21.15.64:0/1079236039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 -- 172.21.15.64:0/1079236039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 msgr2=0x7f45fc162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 --2- 172.21.15.64:0/1079236039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 -- 172.21.15.64:0/1079236039 >> v2:172.21.15.120:3300/0 conn(0x7f45fc140ec0 msgr2=0x7f45fc1416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 --2- 172.21.15.64:0/1079236039 >> v2:172.21.15.120:3300/0 conn(0x7f45fc140ec0 0x7f45fc1416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4602031640 1 -- 172.21.15.64:0/1079236039 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc0b4b50 con 0x7f45fc15e6a0 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.892+0000 7f4601830640 1 --2- 172.21.15.64:0/1079236039 >> v2:172.21.15.120:3300/0 conn(0x7f45fc140ec0 0x7f45fc1416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.893+0000 7f4602031640 1 --2- 172.21.15.64:0/1079236039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc15e6a0 0x7f45fc15eb30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0d40b0 tx=0x7f45f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a1abdad9c67c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.893+0000 7f460082e640 1 -- 172.21.15.64:0/1079236039 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f402ca70 con 0x7f45fc15e6a0 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.893+0000 7f460082e640 1 -- 172.21.15.64:0/1079236039 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f402cc10 con 0x7f45fc15e6a0 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.893+0000 7f460082e640 1 -- 172.21.15.64:0/1079236039 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f40343f0 con 0x7f45fc15e6a0 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 -- 172.21.15.64:0/1079236039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc15e6a0 msgr2=0x7f45fc15eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 --2- 172.21.15.64:0/1079236039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc15e6a0 0x7f45fc15eb30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0d40b0 tx=0x7f45f402bed0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 -- 172.21.15.64:0/1079236039 shutdown_connections 2024-03-07T07:20:00.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 --2- 172.21.15.64:0/1079236039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc15e6a0 0x7f45fc15eb30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 --2- 172.21.15.64:0/1079236039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 --2- 172.21.15.64:0/1079236039 >> v2:172.21.15.120:3300/0 conn(0x7f45fc140ec0 0x7f45fc1416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 -- 172.21.15.64:0/1079236039 >> 172.21.15.64:0/1079236039 conn(0x7f45fc071340 msgr2=0x7f45fc071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:00.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 -- 172.21.15.64:0/1079236039 shutdown_connections 2024-03-07T07:20:00.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.894+0000 7f4603a94640 1 -- 172.21.15.64:0/1079236039 wait complete. 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.895+0000 7f4603a94640 1 Processor -- start 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.895+0000 7f4603a94640 1 -- start start 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4603a94640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc140ec0 0x7f45fc0d2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4603a94640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc15e6a0 0x7f45fc145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4603a94640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4603a94640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc141420 con 0x7f45fc140ec0 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4603a94640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1412a0 con 0x7f45fc15e6a0 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4603a94640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc0b5070 con 0x7f45fc161c90 2024-03-07T07:20:00.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc140ec0 0x7f45fc0d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc140ec0 0x7f45fc0d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35190/0 (socket says 172.21.15.64:35190) 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 -- 172.21.15.64:0/943721389 learned_addr learned my addr 172.21.15.64:0/943721389 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4602031640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f460102f640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc15e6a0 0x7f45fc145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 -- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 msgr2=0x7f45fc146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 -- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc15e6a0 msgr2=0x7f45fc145e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc15e6a0 0x7f45fc145e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4601830640 1 -- 172.21.15.64:0/943721389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc0b8ee0 con 0x7f45fc140ec0 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.896+0000 7f4602031640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.897+0000 7f4601830640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc140ec0 0x7f45fc0d2120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00ebe0 tx=0x7f45ec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.897+0000 7f45f27fc640 1 -- 172.21.15.64:0/943721389 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec017070 con 0x7f45fc140ec0 2024-03-07T07:20:00.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.897+0000 7f45f27fc640 1 -- 172.21.15.64:0/943721389 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec00cc30 con 0x7f45fc140ec0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.897+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc0d2a30 con 0x7f45fc140ec0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.897+0000 7f45f27fc640 1 -- 172.21.15.64:0/943721389 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec013470 con 0x7f45fc140ec0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.897+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc0d2ff0 con 0x7f45fc140ec0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.898+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45fc062c90 con 0x7f45fc140ec0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.898+0000 7f45f27fc640 1 -- 172.21.15.64:0/943721389 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f45ec01a070 con 0x7f45fc140ec0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.898+0000 7f45f27fc640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f45e006b4d0 0x7f45e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.899+0000 7f45f27fc640 1 -- 172.21.15.64:0/943721389 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f45ec08a170 con 0x7f45fc140ec0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.899+0000 7f460102f640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f45e006b4d0 0x7f45e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:00.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.899+0000 7f460102f640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f45e006b4d0 0x7f45e006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f45fc14c540 tx=0x7f45f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:00.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:00.903+0000 7f45f27fc640 1 -- 172.21.15.64:0/943721389 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f45ec059380 con 0x7f45fc140ec0 2024-03-07T07:20:01.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.104+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f45fc0b6000 con 0x7f45fc140ec0 2024-03-07T07:20:01.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.709+0000 7f45f27fc640 1 -- 172.21.15.64:0/943721389 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45ec058c80 con 0x7f45fc140ec0 2024-03-07T07:20:01.709 INFO:teuthology.orchestra.run.smithi064.stderr: removed all blocklist entries 2024-03-07T07:20:01.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f45e006b4d0 msgr2=0x7f45e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:01.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f45e006b4d0 0x7f45e006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f45fc14c540 tx=0x7f45f8008040 comp rx=0 tx=0).stop 2024-03-07T07:20:01.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc140ec0 msgr2=0x7f45fc0d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:01.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc140ec0 0x7f45fc0d2120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00ebe0 tx=0x7f45ec00c520 comp rx=0 tx=0).stop 2024-03-07T07:20:01.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 shutdown_connections 2024-03-07T07:20:01.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f45e006b4d0 0x7f45e006d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45fc161c90 0x7f45fc146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45fc15e6a0 0x7f45fc145e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 --2- 172.21.15.64:0/943721389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45fc140ec0 0x7f45fc0d2120 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 >> 172.21.15.64:0/943721389 conn(0x7f45fc071340 msgr2=0x7f45fc15f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:01.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 shutdown_connections 2024-03-07T07:20:01.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.711+0000 7f4603a94640 1 -- 172.21.15.64:0/943721389 wait complete. 2024-03-07T07:20:01.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-07T07:20:01.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.829+0000 7f91e7b54640 1 Processor -- start 2024-03-07T07:20:01.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.829+0000 7f91e7b54640 1 -- start start 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e7b54640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f91e015c210 0x7f91e0072b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e7b54640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91e0071c30 0x7f91e015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e7b54640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e00a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e7b54640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00cf1d0 con 0x7f91e0071c30 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e7b54640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00ce2e0 con 0x7f91e015c210 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e7b54640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00ce5c0 con 0x7f91e00a9430 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91e0071c30 0x7f91e015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f91e0071c30 0x7f91e015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35202/0 (socket says 172.21.15.64:35202) 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 -- 172.21.15.64:0/2452832664 learned_addr learned my addr 172.21.15.64:0/2452832664 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e58f0640 1 --2- 172.21.15.64:0/2452832664 >> v2:172.21.15.120:3300/0 conn(0x7f91e015c210 0x7f91e0072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:01.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e60f1640 1 --2- 172.21.15.64:0/2452832664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e00a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 -- 172.21.15.64:0/2452832664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 msgr2=0x7f91e00a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 --2- 172.21.15.64:0/2452832664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e00a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 -- 172.21.15.64:0/2452832664 >> v2:172.21.15.120:3300/0 conn(0x7f91e015c210 msgr2=0x7f91e0072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 --2- 172.21.15.64:0/2452832664 >> v2:172.21.15.120:3300/0 conn(0x7f91e015c210 0x7f91e0072b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.830+0000 7f91e50ef640 1 -- 172.21.15.64:0/2452832664 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00ce8a0 con 0x7f91e0071c30 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.831+0000 7f91e50ef640 1 --2- 172.21.15.64:0/2452832664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e0071c30 0x7f91e015c5f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91e00700d0 tx=0x7f91d002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b92f1bf2a3b09a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.831+0000 7f91e48ee640 1 -- 172.21.15.64:0/2452832664 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d002e070 con 0x7f91e0071c30 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.831+0000 7f91e48ee640 1 -- 172.21.15.64:0/2452832664 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d0037070 con 0x7f91e0071c30 2024-03-07T07:20:01.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.831+0000 7f91e48ee640 1 -- 172.21.15.64:0/2452832664 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0031a00 con 0x7f91e0071c30 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 -- 172.21.15.64:0/2452832664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e0071c30 msgr2=0x7f91e015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 --2- 172.21.15.64:0/2452832664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e0071c30 0x7f91e015c5f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91e00700d0 tx=0x7f91d002b880 comp rx=0 tx=0).stop 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 -- 172.21.15.64:0/2452832664 shutdown_connections 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 --2- 172.21.15.64:0/2452832664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e00a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 --2- 172.21.15.64:0/2452832664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e0071c30 0x7f91e015c5f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 --2- 172.21.15.64:0/2452832664 >> v2:172.21.15.120:3300/0 conn(0x7f91e015c210 0x7f91e0072b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 -- 172.21.15.64:0/2452832664 >> 172.21.15.64:0/2452832664 conn(0x7f91e00b21a0 msgr2=0x7f91e00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 -- 172.21.15.64:0/2452832664 shutdown_connections 2024-03-07T07:20:01.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.832+0000 7f91e7b54640 1 -- 172.21.15.64:0/2452832664 wait complete. 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.833+0000 7f91e7b54640 1 Processor -- start 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.833+0000 7f91e7b54640 1 -- start start 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.833+0000 7f91e7b54640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91e0071c30 0x7f91e0153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e7b54640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e01f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e7b54640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e015c210 0x7f91e014cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e7b54640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e015cb30 con 0x7f91e015c210 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e7b54640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cee60 con 0x7f91e0071c30 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e7b54640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e015ccb0 con 0x7f91e00a9430 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e015c210 0x7f91e014cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e015c210 0x7f91e014cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35212/0 (socket says 172.21.15.64:35212) 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 -- 172.21.15.64:0/226357055 learned_addr learned my addr 172.21.15.64:0/226357055 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e50ef640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e01f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e58f0640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91e0071c30 0x7f91e0153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:01.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 -- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 msgr2=0x7f91e01f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e01f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 -- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91e0071c30 msgr2=0x7f91e0153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91e0071c30 0x7f91e0153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 -- 172.21.15.64:0/226357055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e014d110 con 0x7f91e015c210 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e60f1640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e015c210 0x7f91e014cbd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91d8007e90 tx=0x7f91d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91d67fc640 1 -- 172.21.15.64:0/226357055 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8014420 con 0x7f91e015c210 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91d67fc640 1 -- 172.21.15.64:0/226357055 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8019070 con 0x7f91e015c210 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e0153b00 con 0x7f91e015c210 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91d67fc640 1 -- 172.21.15.64:0/226357055 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8014820 con 0x7f91e015c210 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.834+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e01540c0 con 0x7f91e015c210 2024-03-07T07:20:01.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.835+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e0062c90 con 0x7f91e015c210 2024-03-07T07:20:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.836+0000 7f91d67fc640 1 -- 172.21.15.64:0/226357055 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f91d80149c0 con 0x7f91e015c210 2024-03-07T07:20:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.836+0000 7f91d67fc640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f91c406b4d0 0x7f91c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.836+0000 7f91d67fc640 1 -- 172.21.15.64:0/226357055 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f91d808a560 con 0x7f91e015c210 2024-03-07T07:20:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.836+0000 7f91e58f0640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f91c406b4d0 0x7f91c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:01.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.837+0000 7f91e58f0640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f91c406b4d0 0x7f91c406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f91dc006fd0 tx=0x7f91dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:01.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:01.841+0000 7f91d67fc640 1 -- 172.21.15.64:0/226357055 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f91d8059170 con 0x7f91e015c210 2024-03-07T07:20:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.092+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f91e00cc920 con 0x7f91e015c210 2024-03-07T07:20:02.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.092+0000 7f91d67fc640 1 -- 172.21.15.64:0/226357055 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f91d805d210 con 0x7f91e015c210 2024-03-07T07:20:02.092 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:02.092 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:02.092 INFO:teuthology.orchestra.run.smithi064.stdout: "auth_dump": [ 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.a", 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullxsIRCxAA04BZ4y8SMy9m3r4wg+jXXQ==", 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.b", 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aull7hZXFxAARYJ9stnsXcxHajcUfWv9XQ==", 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.093 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.c", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aull3ftcEhAAvnqepJ2iz2sZeEEklpkK6A==", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.094 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mds.d", 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullv/CgHhAAp/jpJGTFbzFhBZQWJaTNFw==", 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.0", 2024-03-07T07:20:02.095 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA9aulloo3+ORAAYhShJsI7HJTH8aSAwHOg7w==", 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.1", 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBAaull+GDlOBAA5veEdAk2Q1YWzJJZo7tEHQ==", 2024-03-07T07:20:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.2", 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBDaullFa1DNxAAdfo86r9Id+f0zqBmYTrzxw==", 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.3", 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBGaullHtAyNRAAmQ3RllAvydkWIV44PJAyAg==", 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.4", 2024-03-07T07:20:02.098 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBNaullHB9RAxAAReqzx+tbp2+wRpM+eyMezg==", 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.5", 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBQaullAtK0ARAAxnNjtvkqTw3ycbUgMpw5+g==", 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.100 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.100 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.6", 2024-03-07T07:20:02.100 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBTaull11tLABAAwQfZk+VvUp1XzZ081/TjNg==", 2024-03-07T07:20:02.100 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.100 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.100 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.100 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.101 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.101 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.101 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "osd.7", 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBVaull5sjqOhAABBkugq1DUhRGzAkMPedDFg==", 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow profile osd", 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile osd", 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.0", 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullg6DUIhAAM9uoMC2xuAIACpfgwkhF9A==", 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.102 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.1", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullyVYaLBAAMjThSrGPA+wqxy1z0EWJzw==", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-03-07T07:20:02.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.2", 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullpla4KBAAMWGeahYoOSfEqSfKAQrh/w==", 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.104 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.3", 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullyYLnMRAAUIhgyc3Dt/FrY6BgPlfpBg==", 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow", 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow r", 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow rw", 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow rwx" 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.admin", 2024-03-07T07:20:02.105 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA5aullzn4jKhAA1Oy5i36HPDPTTabJ0SpRAQ==", 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr": "allow *", 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow *", 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mds", 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBbaullcoJzNhAAjLmq7aOOp0iOMKQO8NHr6w==", 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mds" 2024-03-07T07:20:02.106 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-mgr", 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBbaullvYtzNhAAM/29anQG6FHHMy1hcZAZPg==", 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-osd", 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBbaullmpJzNhAAzJivozdtRA3RxRxpbqrYlQ==", 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-osd" 2024-03-07T07:20:02.107 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd", 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBbaull3phzNhAAPP5qf25zGsKgtsbQaHeuPw==", 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBbaullV59zNhAA5axq2RER52ieDz6Nrpanyg==", 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.108 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "client.bootstrap-rgw", 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQBbaulllKdzNhAAjlTC9JmrAzQxu5i7mTh7Fw==", 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.x", 2024-03-07T07:20:02.109 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullyaSuBxAAK+chNI5foXZF/ihl065jgg==", 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:02.111 INFO:teuthology.orchestra.run.smithi064.stdout: "entity": "mgr.y", 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: "key": "AQA6aullO6iEAxAAywLBdQ3VTqByWJiCPRhFyA==", 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": { 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": "allow *", 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: "mon": "allow profile mgr", 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: "osd": "allow *" 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-03-07T07:20:02.112 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f91c406b4d0 msgr2=0x7f91c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f91c406b4d0 0x7f91c406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f91dc006fd0 tx=0x7f91dc008040 comp rx=0 tx=0).stop 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e015c210 msgr2=0x7f91e014cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e015c210 0x7f91e014cbd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91d8007e90 tx=0x7f91d800e4b0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 shutdown_connections 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f91c406b4d0 0x7f91c406d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91e015c210 0x7f91e014cbd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91e00a9430 0x7f91e01f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 --2- 172.21.15.64:0/226357055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91e0071c30 0x7f91e0153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 >> 172.21.15.64:0/226357055 conn(0x7f91e00b21a0 msgr2=0x7f91e0151010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 shutdown_connections 2024-03-07T07:20:02.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.094+0000 7f91e7b54640 1 -- 172.21.15.64:0/226357055 wait complete. 2024-03-07T07:20:02.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-07T07:20:02.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.218+0000 7f839dcdd640 1 Processor -- start 2024-03-07T07:20:02.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.218+0000 7f839dcdd640 1 -- start start 2024-03-07T07:20:02.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.218+0000 7f839dcdd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f839815f170 0x7f839815e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f839dcdd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8398072cf0 0x7f8398071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f839dcdd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8398071c50 con 0x7f83980d0990 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f839dcdd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839806c3a0 con 0x7f839815f170 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f839dcdd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8398071f30 con 0x7f8398072cf0 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f83977fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f839815f170 0x7f839815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f8397fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f83980d0990 0x7f83981470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56412/0 (socket says 172.21.15.64:56412) 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f8397fff640 1 -- 172.21.15.64:0/1512894613 learned_addr learned my addr 172.21.15.64:0/1512894613 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f83977fe640 1 -- 172.21.15.64:0/1512894613 >> v2:172.21.15.120:3300/0 conn(0x7f8398072cf0 msgr2=0x7f8398071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f83977fe640 1 --2- 172.21.15.64:0/1512894613 >> v2:172.21.15.120:3300/0 conn(0x7f8398072cf0 0x7f8398071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f83977fe640 1 -- 172.21.15.64:0/1512894613 >> v1:172.21.15.64:6789/0 conn(0x7f83980d0990 legacy=0x7f83981470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f83977fe640 1 -- 172.21.15.64:0/1512894613 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8398072210 con 0x7f839815f170 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.219+0000 7f83967fc640 1 -- 172.21.15.64:0/1512894613 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916007826 0 0) 0x7f8398071c50 con 0x7f83980d0990 2024-03-07T07:20:02.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f83977fe640 1 --2- 172.21.15.64:0/1512894613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f839815e650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8398161b70 tx=0x7f838802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715071d66b8367fb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f83967fc640 1 -- 172.21.15.64:0/1512894613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388035020 con 0x7f839815f170 2024-03-07T07:20:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f83967fc640 1 -- 172.21.15.64:0/1512894613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388031640 con 0x7f839815f170 2024-03-07T07:20:02.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f83967fc640 1 -- 172.21.15.64:0/1512894613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838802d040 con 0x7f839815f170 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 -- 172.21.15.64:0/1512894613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 msgr2=0x7f839815e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 --2- 172.21.15.64:0/1512894613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f839815e650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8398161b70 tx=0x7f838802b8f0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 -- 172.21.15.64:0/1512894613 shutdown_connections 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 --2- 172.21.15.64:0/1512894613 >> v2:172.21.15.120:3300/0 conn(0x7f8398072cf0 0x7f8398071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 --2- 172.21.15.64:0/1512894613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f839815e650 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 -- 172.21.15.64:0/1512894613 >> 172.21.15.64:0/1512894613 conn(0x7f839806b480 msgr2=0x7f839806b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 -- 172.21.15.64:0/1512894613 shutdown_connections 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.220+0000 7f839dcdd640 1 -- 172.21.15.64:0/1512894613 wait complete. 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 Processor -- start 2024-03-07T07:20:02.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 -- start start 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8398072cf0 0x7f83980ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83980d0990 0x7f839813a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f83980b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83980d0e20 con 0x7f839815f170 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83980727d0 con 0x7f83980d0990 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.222+0000 7f839dcdd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83980d0fa0 con 0x7f8398072cf0 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f83980b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f83980b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35230/0 (socket says 172.21.15.64:35230) 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 -- 172.21.15.64:0/2606525148 learned_addr learned my addr 172.21.15.64:0/2606525148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:02.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f83977fe640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8398072cf0 0x7f83980ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8396ffd640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83980d0990 0x7f839813a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 -- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8398072cf0 msgr2=0x7f83980ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8398072cf0 0x7f83980ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 -- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83980d0990 msgr2=0x7f839813a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83980d0990 0x7f839813a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 -- 172.21.15.64:0/2606525148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980cd460 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f8397fff640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f83980b29d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f839000c7a0 tx=0x7f839000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f837ffff640 1 -- 172.21.15.64:0/2606525148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390018070 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f837ffff640 1 -- 172.21.15.64:0/2606525148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000a640 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839813a8d0 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f837ffff640 1 -- 172.21.15.64:0/2606525148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839000aab0 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.223+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8398159500 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.224+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8398062c90 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.225+0000 7f837ffff640 1 -- 172.21.15.64:0/2606525148 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f839001b070 con 0x7f839815f170 2024-03-07T07:20:02.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.225+0000 7f837ffff640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f838406b4d0 0x7f838406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.225+0000 7f837ffff640 1 -- 172.21.15.64:0/2606525148 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8390089440 con 0x7f839815f170 2024-03-07T07:20:02.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.225+0000 7f83977fe640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f838406b4d0 0x7f838406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.226+0000 7f83977fe640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f838406b4d0 0x7f838406d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8388036000 tx=0x7f838803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.229+0000 7f837ffff640 1 -- 172.21.15.64:0/2606525148 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8390058610 con 0x7f839815f170 2024-03-07T07:20:02.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.480+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f83980cfa50 con 0x7f839815f170 2024-03-07T07:20:02.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.482+0000 7f837ffff640 1 -- 172.21.15.64:0/2606525148 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8390057f10 con 0x7f839815f170 2024-03-07T07:20:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f838406b4d0 msgr2=0x7f838406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f838406b4d0 0x7f838406d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8388036000 tx=0x7f838803c040 comp rx=0 tx=0).stop 2024-03-07T07:20:02.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 msgr2=0x7f83980b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f83980b29d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f839000c7a0 tx=0x7f839000cc60 comp rx=0 tx=0).stop 2024-03-07T07:20:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 shutdown_connections 2024-03-07T07:20:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f838406b4d0 0x7f838406d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f839815f170 0x7f83980b29d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83980d0990 0x7f839813a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 --2- 172.21.15.64:0/2606525148 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8398072cf0 0x7f83980ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 >> 172.21.15.64:0/2606525148 conn(0x7f839806b480 msgr2=0x7f8398160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:02.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 shutdown_connections 2024-03-07T07:20:02.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.484+0000 7f839dcdd640 1 -- 172.21.15.64:0/2606525148 wait complete. 2024-03-07T07:20:02.495 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-07T07:20:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.598+0000 7fb52c31d640 1 Processor -- start 2024-03-07T07:20:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.598+0000 7fb52c31d640 1 -- start start 2024-03-07T07:20:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52c31d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb5241416d0 0x7fb52415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52c31d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb5240d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52c31d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb524141c60 con 0x7fb524162030 2024-03-07T07:20:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52c31d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5240d3e40 con 0x7fb5241416d0 2024-03-07T07:20:02.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52c31d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240b44b0 con 0x7fb5240d3490 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb5298b8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb5241416d0 0x7fb52415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52a8ba640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb5240d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb5298b8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb5241416d0 0x7fb52415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:59720/0 (socket says 172.21.15.64:59720) 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52a8ba640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb5240d3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:37630/0 (socket says 172.21.15.64:37630) 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb5298b8640 1 -- 172.21.15.64:0/3484114645 learned_addr learned my addr 172.21.15.64:0/3484114645 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb52a0b9640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb524162030 0x7fb524162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:53752/0 (socket says 172.21.15.64:53752) 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb5298b8640 1 -- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 msgr2=0x7fb5240d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.599+0000 7fb5298b8640 1 --2- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb5240d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.600+0000 7fb5298b8640 1 -- 172.21.15.64:0/3484114645 >> v1:172.21.15.120:6789/0 conn(0x7fb524162030 legacy=0x7fb524162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.600+0000 7fb5298b8640 1 -- 172.21.15.64:0/3484114645 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52415ecd0 con 0x7fb5241416d0 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.600+0000 7fb5290b7640 1 -- 172.21.15.64:0/3484114645 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645669638 0 0) 0x7fb524141c60 con 0x7fb524162030 2024-03-07T07:20:02.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.600+0000 7fb52a8ba640 1 --2- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb5240d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.600+0000 7fb5298b8640 1 --2- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 0x7fb52415e680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb514009870 tx=0x7fb51402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a268d6829816420 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb5290b7640 1 -- 172.21.15.64:0/3484114645 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514034070 con 0x7fb5241416d0 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb5290b7640 1 -- 172.21.15.64:0/3484114645 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514030840 con 0x7fb5241416d0 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb5290b7640 1 -- 172.21.15.64:0/3484114645 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514030c10 con 0x7fb5241416d0 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 -- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 msgr2=0x7fb52415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 0x7fb52415e680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb514009870 tx=0x7fb51402b980 comp rx=0 tx=0).stop 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 -- 172.21.15.64:0/3484114645 shutdown_connections 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb5240d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3484114645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 0x7fb52415e680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 -- 172.21.15.64:0/3484114645 >> 172.21.15.64:0/3484114645 conn(0x7fb524070400 msgr2=0x7fb5240d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 -- 172.21.15.64:0/3484114645 shutdown_connections 2024-03-07T07:20:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.601+0000 7fb52c31d640 1 -- 172.21.15.64:0/3484114645 wait complete. 2024-03-07T07:20:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.602+0000 7fb52c31d640 1 Processor -- start 2024-03-07T07:20:02.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.602+0000 7fb52c31d640 1 -- start start 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52c31d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb524158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52c31d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 0x7fb5241457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52c31d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524162030 0x7fb52415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52c31d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524162b20 con 0x7fb524162030 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52c31d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52415f290 con 0x7fb5241416d0 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52c31d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52415f570 con 0x7fb5240d3490 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524162030 0x7fb52415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524162030 0x7fb52415d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35240/0 (socket says 172.21.15.64:35240) 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 -- 172.21.15.64:0/3458353976 learned_addr learned my addr 172.21.15.64:0/3458353976 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb5298b8640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 0x7fb5241457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a0b9640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb524158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 -- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 msgr2=0x7fb524158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb524158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 -- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 msgr2=0x7fb5241457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 0x7fb5241457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.603+0000 7fb52a8ba640 1 -- 172.21.15.64:0/3458353976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5241461f0 con 0x7fb524162030 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.604+0000 7fb52a8ba640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524162030 0x7fb52415d710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00efc0 tx=0x7fb51c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.604+0000 7fb51affd640 1 -- 172.21.15.64:0/3458353976 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c018030 con 0x7fb524162030 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.604+0000 7fb51affd640 1 -- 172.21.15.64:0/3458353976 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c00cdf0 con 0x7fb524162030 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.604+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5241464d0 con 0x7fb524162030 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.604+0000 7fb51affd640 1 -- 172.21.15.64:0/3458353976 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c003bb0 con 0x7fb524162030 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.604+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb524146a90 con 0x7fb524162030 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.605+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb524062c90 con 0x7fb524162030 2024-03-07T07:20:02.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.605+0000 7fb51affd640 1 -- 172.21.15.64:0/3458353976 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb51c01b070 con 0x7fb524162030 2024-03-07T07:20:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.605+0000 7fb51affd640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb50806b4d0 0x7fb50806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.606+0000 7fb51affd640 1 -- 172.21.15.64:0/3458353976 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb51c08a750 con 0x7fb524162030 2024-03-07T07:20:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.606+0000 7fb52a0b9640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb50806b4d0 0x7fb50806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.606+0000 7fb52a0b9640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb50806b4d0 0x7fb50806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb520000f60 tx=0x7fb520007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.610+0000 7fb51affd640 1 -- 172.21.15.64:0/3458353976 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb51c010040 con 0x7fb524162030 2024-03-07T07:20:02.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.860+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fb524071950 con 0x7fb524162030 2024-03-07T07:20:02.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.862+0000 7fb51affd640 1 -- 172.21.15.64:0/3458353976 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb51c0592d0 con 0x7fb524162030 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb50806b4d0 msgr2=0x7fb50806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb50806b4d0 0x7fb50806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb520000f60 tx=0x7fb520007d50 comp rx=0 tx=0).stop 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524162030 msgr2=0x7fb52415d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524162030 0x7fb52415d710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00efc0 tx=0x7fb51c00c490 comp rx=0 tx=0).stop 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 shutdown_connections 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb50806b4d0 0x7fb50806d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb524162030 0x7fb52415d710 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5241416d0 0x7fb5241457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 --2- 172.21.15.64:0/3458353976 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5240d3490 0x7fb524158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 >> 172.21.15.64:0/3458353976 conn(0x7fb524070400 msgr2=0x7fb5240d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:02.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 shutdown_connections 2024-03-07T07:20:02.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.864+0000 7fb52c31d640 1 -- 172.21.15.64:0/3458353976 wait complete. 2024-03-07T07:20:02.874 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-07T07:20:02.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.977+0000 7ff607c87640 1 Processor -- start 2024-03-07T07:20:02.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.977+0000 7ff607c87640 1 -- start start 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff607c87640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6001416d0 0x7ff60015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff607c87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6000d3490 0x7ff6000d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff607c87640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff600141c60 con 0x7ff600162030 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff607c87640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000d3e40 con 0x7ff6001416d0 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff607c87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000b44b0 con 0x7ff6000d3490 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff605222640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6001416d0 0x7ff60015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff605222640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7ff6001416d0 0x7ff60015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35246/0 (socket says 172.21.15.64:35246) 2024-03-07T07:20:02.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff605222640 1 -- 172.21.15.64:0/1238242185 learned_addr learned my addr 172.21.15.64:0/1238242185 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff606224640 1 --2- 172.21.15.64:0/1238242185 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6000d3490 0x7ff6000d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff605222640 1 -- 172.21.15.64:0/1238242185 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6000d3490 msgr2=0x7ff6000d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff605222640 1 --2- 172.21.15.64:0/1238242185 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6000d3490 0x7ff6000d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff605222640 1 -- 172.21.15.64:0/1238242185 >> v1:172.21.15.120:6789/0 conn(0x7ff600162030 legacy=0x7ff600162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.978+0000 7ff605222640 1 -- 172.21.15.64:0/1238242185 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60015ecd0 con 0x7ff6001416d0 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.979+0000 7ff605222640 1 --2- 172.21.15.64:0/1238242185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6001416d0 0x7ff60015e680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009870 tx=0x7ff5f002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5733d27339ad4fef server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.979+0000 7ff604a21640 1 -- 172.21.15.64:0/1238242185 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0034070 con 0x7ff6001416d0 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.979+0000 7ff604a21640 1 -- 172.21.15.64:0/1238242185 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f0030840 con 0x7ff6001416d0 2024-03-07T07:20:02.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.979+0000 7ff604a21640 1 -- 172.21.15.64:0/1238242185 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0030c10 con 0x7ff6001416d0 2024-03-07T07:20:02.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 -- 172.21.15.64:0/1238242185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6001416d0 msgr2=0x7ff60015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 --2- 172.21.15.64:0/1238242185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6001416d0 0x7ff60015e680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009870 tx=0x7ff5f002b980 comp rx=0 tx=0).stop 2024-03-07T07:20:02.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 -- 172.21.15.64:0/1238242185 shutdown_connections 2024-03-07T07:20:02.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 --2- 172.21.15.64:0/1238242185 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6000d3490 0x7ff6000d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 --2- 172.21.15.64:0/1238242185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6001416d0 0x7ff60015e680 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 -- 172.21.15.64:0/1238242185 >> 172.21.15.64:0/1238242185 conn(0x7ff600070400 msgr2=0x7ff6000d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 -- 172.21.15.64:0/1238242185 shutdown_connections 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.980+0000 7ff607c87640 1 -- 172.21.15.64:0/1238242185 wait complete. 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.981+0000 7ff607c87640 1 Processor -- start 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.981+0000 7ff607c87640 1 -- start start 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff607c87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6000d3490 0x7ff600160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff607c87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6001416d0 0x7ff6001572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff607c87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff600162030 0x7ff6001f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff607c87640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600162b20 con 0x7ff6000d3490 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff607c87640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015f290 con 0x7ff600162030 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff607c87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015f570 con 0x7ff6001416d0 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6000d3490 0x7ff600160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6000d3490 0x7ff600160bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35256/0 (socket says 172.21.15.64:35256) 2024-03-07T07:20:02.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 -- 172.21.15.64:0/180654250 learned_addr learned my addr 172.21.15.64:0/180654250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605222640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6001416d0 0x7ff6001572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff606224640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff600162030 0x7ff6001f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 -- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6001416d0 msgr2=0x7ff6001572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6001416d0 0x7ff6001572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 -- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff600162030 msgr2=0x7ff6001f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff600162030 0x7ff6001f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 -- 172.21.15.64:0/180654250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600147770 con 0x7ff6000d3490 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605222640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6001416d0 0x7ff6001572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff605a23640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6000d3490 0x7ff600160bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c400 tx=0x7ff5fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff5f67fc640 1 -- 172.21.15.64:0/180654250 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0052d0 con 0x7ff6000d3490 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.982+0000 7ff5f67fc640 1 -- 172.21.15.64:0/180654250 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc005490 con 0x7ff6000d3490 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.983+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6001471b0 con 0x7ff6000d3490 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.983+0000 7ff5f67fc640 1 -- 172.21.15.64:0/180654250 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc007260 con 0x7ff6000d3490 2024-03-07T07:20:02.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.983+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff600145820 con 0x7ff6000d3490 2024-03-07T07:20:02.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.984+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600062c90 con 0x7ff6000d3490 2024-03-07T07:20:02.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.984+0000 7ff5f67fc640 1 -- 172.21.15.64:0/180654250 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff5fc007400 con 0x7ff6000d3490 2024-03-07T07:20:02.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.984+0000 7ff5f67fc640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:02.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.984+0000 7ff5f67fc640 1 -- 172.21.15.64:0/180654250 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5fc089370 con 0x7ff6000d3490 2024-03-07T07:20:02.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.985+0000 7ff605222640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:02.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.985+0000 7ff605222640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff5e406b4d0 0x7ff5e406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0036000 tx=0x7ff5f003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:02.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:02.989+0000 7ff5f67fc640 1 -- 172.21.15.64:0/180654250 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff5fc0585d0 con 0x7ff6000d3490 2024-03-07T07:20:03.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.237+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ff6000b5210 con 0x7ff6000d3490 2024-03-07T07:20:03.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.239+0000 7ff5f67fc640 1 -- 172.21.15.64:0/180654250 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff5fc057ed0 con 0x7ff6000d3490 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.241+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff5e406b4d0 msgr2=0x7ff5e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff5e406b4d0 0x7ff5e406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0036000 tx=0x7ff5f003b040 comp rx=0 tx=0).stop 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6000d3490 msgr2=0x7ff600160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6000d3490 0x7ff600160bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c400 tx=0x7ff5fc00c8c0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 shutdown_connections 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff5e406b4d0 0x7ff5e406d990 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0036000 tx=0x7ff5f003b040 comp rx=0 tx=0).stop 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff600162030 0x7ff6001f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6001416d0 0x7ff6001572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 --2- 172.21.15.64:0/180654250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6000d3490 0x7ff600160bc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 >> 172.21.15.64:0/180654250 conn(0x7ff600070400 msgr2=0x7ff6000b7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:03.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 shutdown_connections 2024-03-07T07:20:03.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.242+0000 7ff607c87640 1 -- 172.21.15.64:0/180654250 wait complete. 2024-03-07T07:20:03.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-07T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.355+0000 7f05dcb10640 1 Processor -- start 2024-03-07T07:20:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.355+0000 7f05dcb10640 1 -- start start 2024-03-07T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.355+0000 7f05dcb10640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b7360 0x7f05d80b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.355+0000 7f05dcb10640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05d80ceac0 0x7f05d80cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.355+0000 7f05dcb10640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d80b5830 con 0x7f05d80b4f10 2024-03-07T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05dcb10640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d80d0970 con 0x7f05d80ceac0 2024-03-07T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05dcb10640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d80d0bc0 con 0x7f05d80b7360 2024-03-07T07:20:03.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d6d9d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f05d80ceac0 0x7f05d80cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d659c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f05d80b4f10 0x7f05d815cef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56422/0 (socket says 172.21.15.64:56422) 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d659c640 1 -- 172.21.15.64:0/2413332952 learned_addr learned my addr 172.21.15.64:0/2413332952 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d6d9d640 1 -- 172.21.15.64:0/2413332952 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b7360 msgr2=0x7f05d80b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d5d9b640 1 --2- 172.21.15.64:0/2413332952 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b7360 0x7f05d80b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d6d9d640 1 --2- 172.21.15.64:0/2413332952 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b7360 0x7f05d80b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d559a640 1 -- 172.21.15.64:0/2413332952 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266597317 0 0) 0x7f05d80b5830 con 0x7f05d80b4f10 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d6d9d640 1 -- 172.21.15.64:0/2413332952 >> v1:172.21.15.64:6789/0 conn(0x7f05d80b4f10 legacy=0x7f05d815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d6d9d640 1 -- 172.21.15.64:0/2413332952 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d80d0ea0 con 0x7f05d80ceac0 2024-03-07T07:20:03.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.356+0000 7f05d5d9b640 1 --2- 172.21.15.64:0/2413332952 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b7360 0x7f05d80b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05d6d9d640 1 --2- 172.21.15.64:0/2413332952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80ceac0 0x7f05d80cef70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009d90 tx=0x7f05c802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892b09e0ac8321e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05d559a640 1 -- 172.21.15.64:0/2413332952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8039070 con 0x7f05d80ceac0 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05d559a640 1 -- 172.21.15.64:0/2413332952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c802cc50 con 0x7f05d80ceac0 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05d559a640 1 -- 172.21.15.64:0/2413332952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8034440 con 0x7f05d80ceac0 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05dcb10640 1 -- 172.21.15.64:0/2413332952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80ceac0 msgr2=0x7f05d80cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05dcb10640 1 --2- 172.21.15.64:0/2413332952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80ceac0 0x7f05d80cef70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009d90 tx=0x7f05c802bea0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05dcb10640 1 -- 172.21.15.64:0/2413332952 shutdown_connections 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.357+0000 7f05dcb10640 1 --2- 172.21.15.64:0/2413332952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80ceac0 0x7f05d80cef70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.358+0000 7f05dcb10640 1 --2- 172.21.15.64:0/2413332952 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b7360 0x7f05d80b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.358+0000 7f05dcb10640 1 -- 172.21.15.64:0/2413332952 >> 172.21.15.64:0/2413332952 conn(0x7f05d806b480 msgr2=0x7f05d806b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:03.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.358+0000 7f05dcb10640 1 -- 172.21.15.64:0/2413332952 shutdown_connections 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.358+0000 7f05dcb10640 1 -- 172.21.15.64:0/2413332952 wait complete. 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.359+0000 7f05dcb10640 1 Processor -- start 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.359+0000 7f05dcb10640 1 -- start start 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.359+0000 7f05dcb10640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b4f10 0x7f05d81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.359+0000 7f05dcb10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80b7360 0x7f05d80c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.359+0000 7f05dcb10640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d80ceac0 0x7f05d81579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05dcb10640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d80d1740 con 0x7f05d80b7360 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05dcb10640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d80d1460 con 0x7f05d80ceac0 2024-03-07T07:20:03.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05dcb10640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8159d10 con 0x7f05d80b4f10 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80b7360 0x7f05d80c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80b7360 0x7f05d80c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35272/0 (socket says 172.21.15.64:35272) 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 -- 172.21.15.64:0/1058783938 learned_addr learned my addr 172.21.15.64:0/1058783938 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d659c640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b4f10 0x7f05d81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d6d9d640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d80ceac0 0x7f05d81579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 -- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b4f10 msgr2=0x7f05d81498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b4f10 0x7f05d81498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 -- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d80ceac0 msgr2=0x7f05d81579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d80ceac0 0x7f05d81579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 -- 172.21.15.64:0/1058783938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d80c8a50 con 0x7f05d80b7360 2024-03-07T07:20:03.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05d5d9b640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80b7360 0x7f05d80c7f90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c820 tx=0x7f05c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05ceffd640 1 -- 172.21.15.64:0/1058783938 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0015070 con 0x7f05d80b7360 2024-03-07T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05ceffd640 1 -- 172.21.15.64:0/1058783938 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c0007a30 con 0x7f05d80b7360 2024-03-07T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d80c8d30 con 0x7f05d80b7360 2024-03-07T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05ceffd640 1 -- 172.21.15.64:0/1058783938 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0007e00 con 0x7f05d80b7360 2024-03-07T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.360+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d8149de0 con 0x7f05d80b7360 2024-03-07T07:20:03.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.361+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d8062c90 con 0x7f05d80b7360 2024-03-07T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.362+0000 7f05ceffd640 1 -- 172.21.15.64:0/1058783938 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f05c0018070 con 0x7f05d80b7360 2024-03-07T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.362+0000 7f05ceffd640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f05b806b4d0 0x7f05b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.362+0000 7f05ceffd640 1 -- 172.21.15.64:0/1058783938 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05c0089e60 con 0x7f05d80b7360 2024-03-07T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.362+0000 7f05d659c640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f05b806b4d0 0x7f05b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.363+0000 7f05d659c640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f05b806b4d0 0x7f05b806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05d0000f60 tx=0x7f05d0007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:03.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.366+0000 7f05ceffd640 1 -- 172.21.15.64:0/1058783938 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f05c0059030 con 0x7f05d80b7360 2024-03-07T07:20:03.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.615+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f05d80b36e0 con 0x7f05d80b7360 2024-03-07T07:20:03.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.618+0000 7f05ceffd640 1 -- 172.21.15.64:0/1058783938 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f05c0058930 con 0x7f05d80b7360 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.619+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f05b806b4d0 msgr2=0x7f05b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.619+0000 7f05dcb10640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f05b806b4d0 0x7f05b806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05d0000f60 tx=0x7f05d0007b50 comp rx=0 tx=0).stop 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80b7360 msgr2=0x7f05d80c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80b7360 0x7f05d80c7f90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c820 tx=0x7f05c000cce0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 shutdown_connections 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f05b806b4d0 0x7f05b806d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05d80b7360 0x7f05d80c7f90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05d80ceac0 0x7f05d81579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 --2- 172.21.15.64:0/1058783938 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f05d80b4f10 0x7f05d81498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 >> 172.21.15.64:0/1058783938 conn(0x7f05d806b480 msgr2=0x7f05d806b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:03.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 shutdown_connections 2024-03-07T07:20:03.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.620+0000 7f05dcb10640 1 -- 172.21.15.64:0/1058783938 wait complete. 2024-03-07T07:20:03.630 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-07T07:20:03.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.733+0000 7f9da7ab6640 1 Processor -- start 2024-03-07T07:20:03.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.733+0000 7f9da7ab6640 1 -- start start 2024-03-07T07:20:03.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da7ab6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9da00b4f10 0x7f9da015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da7ab6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da00b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da7ab6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da015d430 con 0x7f9da015fa60 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da7ab6640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da00d0970 con 0x7f9da00b4f10 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da7ab6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00b5830 con 0x7f9da00b7360 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da5852640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f9da00b4f10 0x7f9da015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da6053640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f9da015fa60 0x7f9da015ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56432/0 (socket says 172.21.15.64:56432) 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da6053640 1 -- 172.21.15.64:0/1240204533 learned_addr learned my addr 172.21.15.64:0/1240204533 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da5051640 1 --2- 172.21.15.64:0/1240204533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da00b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da5852640 1 -- 172.21.15.64:0/1240204533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 msgr2=0x7f9da00b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da5852640 1 --2- 172.21.15.64:0/1240204533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da00b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da5852640 1 -- 172.21.15.64:0/1240204533 >> v1:172.21.15.64:6789/0 conn(0x7f9da015fa60 legacy=0x7f9da015ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da5852640 1 -- 172.21.15.64:0/1240204533 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00d0bc0 con 0x7f9da00b4f10 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da4850640 1 -- 172.21.15.64:0/1240204533 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719652333 0 0) 0x7f9d90001490 con 0x7f9da015fa60 2024-03-07T07:20:03.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.734+0000 7f9da5051640 1 --2- 172.21.15.64:0/1240204533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da00b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.735+0000 7f9da5852640 1 --2- 172.21.15.64:0/1240204533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da00b4f10 0x7f9da015cef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9d980097a0 tx=0x7f9d9802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6c248a17f631d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.735+0000 7f9da4850640 1 -- 172.21.15.64:0/1240204533 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98035020 con 0x7f9da00b4f10 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.735+0000 7f9da4850640 1 -- 172.21.15.64:0/1240204533 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d98031830 con 0x7f9da00b4f10 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.735+0000 7f9da4850640 1 -- 172.21.15.64:0/1240204533 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98031c00 con 0x7f9da00b4f10 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 -- 172.21.15.64:0/1240204533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da00b4f10 msgr2=0x7f9da015cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/1240204533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da00b4f10 0x7f9da015cef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9d980097a0 tx=0x7f9d9802b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 -- 172.21.15.64:0/1240204533 shutdown_connections 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/1240204533 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da00b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/1240204533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da00b4f10 0x7f9da015cef0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 -- 172.21.15.64:0/1240204533 >> 172.21.15.64:0/1240204533 conn(0x7f9da006b480 msgr2=0x7f9da006b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:03.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 -- 172.21.15.64:0/1240204533 shutdown_connections 2024-03-07T07:20:03.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.736+0000 7f9da7ab6640 1 -- 172.21.15.64:0/1240204533 wait complete. 2024-03-07T07:20:03.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.737+0000 7f9da7ab6640 1 Processor -- start 2024-03-07T07:20:03.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.737+0000 7f9da7ab6640 1 -- start start 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da7ab6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da00b4f10 0x7f9da00c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da7ab6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da7ab6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da015fa60 0x7f9da0157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da7ab6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00d1460 con 0x7f9da015fa60 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da7ab6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00d1180 con 0x7f9da00b4f10 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da7ab6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00d1740 con 0x7f9da00b7360 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da015fa60 0x7f9da0157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da015fa60 0x7f9da0157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35288/0 (socket says 172.21.15.64:35288) 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 -- 172.21.15.64:0/2314030296 learned_addr learned my addr 172.21.15.64:0/2314030296 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da5051640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da5852640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da00b4f10 0x7f9da00c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 -- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 msgr2=0x7f9da014afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da014afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 -- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da00b4f10 msgr2=0x7f9da00c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da00b4f10 0x7f9da00c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da6053640 1 -- 172.21.15.64:0/2314030296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da014f2f0 con 0x7f9da015fa60 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da5051640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.738+0000 7f9da5852640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da00b4f10 0x7f9da00c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.739+0000 7f9da6053640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da015fa60 0x7f9da0157870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c7d0 tx=0x7f9d9000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.739+0000 7f9d9e7fc640 1 -- 172.21.15.64:0/2314030296 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90018070 con 0x7f9da015fa60 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.739+0000 7f9d9e7fc640 1 -- 172.21.15.64:0/2314030296 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000a640 con 0x7f9da015fa60 2024-03-07T07:20:03.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.739+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da0151360 con 0x7f9da015fa60 2024-03-07T07:20:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.739+0000 7f9d9e7fc640 1 -- 172.21.15.64:0/2314030296 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000aab0 con 0x7f9da015fa60 2024-03-07T07:20:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.739+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da014b500 con 0x7f9da015fa60 2024-03-07T07:20:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.740+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da0062c90 con 0x7f9da015fa60 2024-03-07T07:20:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.740+0000 7f9d9e7fc640 1 -- 172.21.15.64:0/2314030296 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9d9001b070 con 0x7f9da015fa60 2024-03-07T07:20:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.741+0000 7f9d9e7fc640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9d8c06b4d0 0x7f9d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:03.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.741+0000 7f9d9e7fc640 1 -- 172.21.15.64:0/2314030296 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d90089530 con 0x7f9da015fa60 2024-03-07T07:20:03.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.741+0000 7f9da5852640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9d8c06b4d0 0x7f9d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:03.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.741+0000 7f9da5852640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9d8c06b4d0 0x7f9d8c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d9803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:03.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.745+0000 7f9d9e7fc640 1 -- 172.21.15.64:0/2314030296 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9da015d430 con 0x7f9da015fa60 2024-03-07T07:20:03.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.998+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9da015be30 con 0x7f9da015fa60 2024-03-07T07:20:03.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:03.999+0000 7f9d9e7fc640 1 -- 172.21.15.64:0/2314030296 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9d90058700 con 0x7f9da015fa60 2024-03-07T07:20:04.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9d8c06b4d0 msgr2=0x7f9d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9d8c06b4d0 0x7f9d8c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d9803c040 comp rx=0 tx=0).stop 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da015fa60 msgr2=0x7f9da0157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da015fa60 0x7f9da0157870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c7d0 tx=0x7f9d9000cc90 comp rx=0 tx=0).stop 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 shutdown_connections 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9d8c06b4d0 0x7f9d8c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9da015fa60 0x7f9da0157870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9da00b7360 0x7f9da014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 --2- 172.21.15.64:0/2314030296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9da00b4f10 0x7f9da00c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 >> 172.21.15.64:0/2314030296 conn(0x7f9da006b480 msgr2=0x7f9da006b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 shutdown_connections 2024-03-07T07:20:04.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.002+0000 7f9da7ab6640 1 -- 172.21.15.64:0/2314030296 wait complete. 2024-03-07T07:20:04.013 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-07T07:20:04.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.115+0000 7f4fa07c9640 1 Processor -- start 2024-03-07T07:20:04.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.115+0000 7f4fa07c9640 1 -- start start 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4fa07c9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f98162030 0x7f4f9815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4fa07c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4f981416d0 0x7f4f98162410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4fa07c9640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f98141c60 con 0x7f4f98162950 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4fa07c9640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980b44b0 con 0x7f4f98162030 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4fa07c9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9815ecd0 con 0x7f4f981416d0 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9e565640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4f98162030 0x7f4f9815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9ed66640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4f98162950 0x7f4f980d3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56442/0 (socket says 172.21.15.64:56442) 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9ed66640 1 -- 172.21.15.64:0/2318933134 learned_addr learned my addr 172.21.15.64:0/2318933134 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9e565640 1 -- 172.21.15.64:0/2318933134 >> v2:172.21.15.120:3300/0 conn(0x7f4f981416d0 msgr2=0x7f4f98162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9dd64640 1 --2- 172.21.15.64:0/2318933134 >> v2:172.21.15.120:3300/0 conn(0x7f4f981416d0 0x7f4f98162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9e565640 1 --2- 172.21.15.64:0/2318933134 >> v2:172.21.15.120:3300/0 conn(0x7f4f981416d0 0x7f4f98162410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9e565640 1 -- 172.21.15.64:0/2318933134 >> v1:172.21.15.64:6789/0 conn(0x7f4f98162950 legacy=0x7f4f980d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9d563640 1 -- 172.21.15.64:0/2318933134 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317981164 0 0) 0x7f4f98141c60 con 0x7f4f98162950 2024-03-07T07:20:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.116+0000 7f4f9e565640 1 -- 172.21.15.64:0/2318933134 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9815efb0 con 0x7f4f98162030 2024-03-07T07:20:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4f9e565640 1 --2- 172.21.15.64:0/2318933134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f9815e680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4f98074e10 tx=0x7f4f9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25d7b33c513ab1f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4f9d563640 1 -- 172.21.15.64:0/2318933134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94035020 con 0x7f4f98162030 2024-03-07T07:20:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4f9d563640 1 -- 172.21.15.64:0/2318933134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94031830 con 0x7f4f98162030 2024-03-07T07:20:04.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4f9d563640 1 -- 172.21.15.64:0/2318933134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94031c00 con 0x7f4f98162030 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4fa07c9640 1 -- 172.21.15.64:0/2318933134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 msgr2=0x7f4f9815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/2318933134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f9815e680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4f98074e10 tx=0x7f4f9402b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4fa07c9640 1 -- 172.21.15.64:0/2318933134 shutdown_connections 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.117+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/2318933134 >> v2:172.21.15.120:3300/0 conn(0x7f4f981416d0 0x7f4f98162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.118+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/2318933134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f9815e680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.118+0000 7f4fa07c9640 1 -- 172.21.15.64:0/2318933134 >> 172.21.15.64:0/2318933134 conn(0x7f4f98070400 msgr2=0x7f4f980d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.118+0000 7f4fa07c9640 1 -- 172.21.15.64:0/2318933134 shutdown_connections 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.118+0000 7f4fa07c9640 1 -- 172.21.15.64:0/2318933134 wait complete. 2024-03-07T07:20:04.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 Processor -- start 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 -- start start 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f981416d0 0x7f4f98147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f981548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98162950 0x7f4f981567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980d3ba0 con 0x7f4f98162030 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9815f570 con 0x7f4f98162950 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.119+0000 7f4fa07c9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980d3d20 con 0x7f4f981416d0 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f981548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f981548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35304/0 (socket says 172.21.15.64:35304) 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 -- 172.21.15.64:0/1236029117 learned_addr learned my addr 172.21.15.64:0/1236029117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9ed66640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98162950 0x7f4f981567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9e565640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f981416d0 0x7f4f98147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 -- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f981416d0 msgr2=0x7f4f98147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f981416d0 0x7f4f98147490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 -- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98162950 msgr2=0x7f4f981567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98162950 0x7f4f981567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 -- 172.21.15.64:0/1236029117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980d3620 con 0x7f4f98162030 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f9dd64640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f981548a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800c7f0 tx=0x7f4f8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f8effd640 1 -- 172.21.15.64:0/1236029117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88015070 con 0x7f4f98162030 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f8effd640 1 -- 172.21.15.64:0/1236029117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f88004220 con 0x7f4f98162030 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f980d3900 con 0x7f4f98162030 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4f8effd640 1 -- 172.21.15.64:0/1236029117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88007620 con 0x7f4f98162030 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.120+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f98144ca0 con 0x7f4f98162030 2024-03-07T07:20:04.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.121+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f98062c90 con 0x7f4f98162030 2024-03-07T07:20:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.122+0000 7f4f8effd640 1 -- 172.21.15.64:0/1236029117 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4f880079d0 con 0x7f4f98162030 2024-03-07T07:20:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.122+0000 7f4f8effd640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.122+0000 7f4f8effd640 1 -- 172.21.15.64:0/1236029117 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f88089e60 con 0x7f4f98162030 2024-03-07T07:20:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.122+0000 7f4f9e565640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.123+0000 7f4f9e565640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f94036000 tx=0x7f4f94008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.126+0000 7f4f8effd640 1 -- 172.21.15.64:0/1236029117 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4f88059030 con 0x7f4f98162030 2024-03-07T07:20:04.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.382+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f4f9815e6a0 con 0x7f4f98162030 2024-03-07T07:20:04.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.384+0000 7f4f8effd640 1 -- 172.21.15.64:0/1236029117 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4f88058930 con 0x7f4f98162030 2024-03-07T07:20:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.386+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7c06b4d0 msgr2=0x7f4f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f94036000 tx=0x7f4f94008030 comp rx=0 tx=0).stop 2024-03-07T07:20:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 msgr2=0x7f4f981548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f981548a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800c7f0 tx=0x7f4f8800ccb0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 shutdown_connections 2024-03-07T07:20:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f98162030 0x7f4f981548a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f98162950 0x7f4f981567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 --2- 172.21.15.64:0/1236029117 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f981416d0 0x7f4f98147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 >> 172.21.15.64:0/1236029117 conn(0x7f4f98070400 msgr2=0x7f4f98142700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 shutdown_connections 2024-03-07T07:20:04.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.387+0000 7f4fa07c9640 1 -- 172.21.15.64:0/1236029117 wait complete. 2024-03-07T07:20:04.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:20:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.499+0000 7f04f9667640 1 Processor -- start 2024-03-07T07:20:04.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.499+0000 7f04f9667640 1 -- start start 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.500+0000 7f04f9667640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 0x7f04f406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.500+0000 7f04f9667640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04f40d2c10 0x7f04f406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.500+0000 7f04f9667640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f40d1e70 con 0x7f04f4147810 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.500+0000 7f04f9667640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f4147650 con 0x7f04f40d2c10 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.500+0000 7f04f9667640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40b7520 con 0x7f04f4147270 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.500+0000 7f04f37fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f04f4147810 0x7f04f4162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56458/0 (socket says 172.21.15.64:56458) 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.500+0000 7f04f37fe640 1 -- 172.21.15.64:0/8153002 learned_addr learned my addr 172.21.15.64:0/8153002 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375192645 0 0) 0x7f04f40d1e70 con 0x7f04f4147810 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f27fc640 1 --2- 172.21.15.64:0/8153002 >> v2:172.21.15.120:3300/0 conn(0x7f04f40d2c10 0x7f04f406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e0003520 con 0x7f04f4147810 2024-03-07T07:20:04.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 0x7f04f406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517053788 0 0) 0x7f04e0003520 con 0x7f04f4147810 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 msgr2=0x7f04f406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 0x7f04f406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 >> v2:172.21.15.120:3300/0 conn(0x7f04f40d2c10 msgr2=0x7f04f406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 --2- 172.21.15.64:0/8153002 >> v2:172.21.15.120:3300/0 conn(0x7f04f40d2c10 0x7f04f406ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40d2130 con 0x7f04f4147810 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f04ec001670 con 0x7f04f4147810 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 >> v1:172.21.15.64:6789/0 conn(0x7f04f4147810 legacy=0x7f04f4162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e00039b0 0x7f04e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 0x7f04e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0004d40 0x7f04e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4147650 con 0x7f04e00039b0 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40b7520 con 0x7f04e0004d40 2024-03-07T07:20:04.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e000b9c0 con 0x7f04f4147270 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e00039b0 0x7f04e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f27fc640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 0x7f04e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f37fe640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0004d40 0x7f04e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f2ffd640 1 -- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 msgr2=0x7f04e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04f4147270 0x7f04e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f2ffd640 1 -- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0004d40 msgr2=0x7f04e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04e0004d40 0x7f04e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.501+0000 7f04f2ffd640 1 -- 172.21.15.64:0/8153002 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e0003520 con 0x7f04e00039b0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.502+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e00039b0 0x7f04e0003dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009fc0 tx=0x7f04e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba35200f02c84e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.502+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4039070 con 0x7f04e00039b0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.502+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e402cc20 con 0x7f04e00039b0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.502+0000 7f04f1ffb640 1 -- 172.21.15.64:0/8153002 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4009c10 con 0x7f04e00039b0 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.503+0000 7f04f9667640 1 -- 172.21.15.64:0/8153002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e00039b0 msgr2=0x7f04e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.503+0000 7f04f9667640 1 --2- 172.21.15.64:0/8153002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e00039b0 0x7f04e0003dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009fc0 tx=0x7f04e0005830 comp rx=0 tx=0).stop 2024-03-07T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.503+0000 7f04f37fe640 1 -- 172.21.15.64:0/8153002 reap_dead start 2024-03-07T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.503+0000 7f04f9667640 1 -- 172.21.15.64:0/8153002 shutdown_connections 2024-03-07T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.503+0000 7f04f9667640 1 -- 172.21.15.64:0/8153002 >> 172.21.15.64:0/8153002 conn(0x7f04f406d4c0 msgr2=0x7f04f4066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.503+0000 7f04f9667640 1 -- 172.21.15.64:0/8153002 shutdown_connections 2024-03-07T07:20:04.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.503+0000 7f04f9667640 1 -- 172.21.15.64:0/8153002 wait complete. 2024-03-07T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.504+0000 7f04f9667640 1 Processor -- start 2024-03-07T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.504+0000 7f04f9667640 1 -- start start 2024-03-07T07:20:04.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f9667640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04e00039b0 0x7f04f40b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f9667640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f40cc540 0x7f04f40cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f9667640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04f4135320 0x7f04f4161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f9667640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40b71d0 con 0x7f04f40cc540 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f9667640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f40d26f0 con 0x7f04f4135320 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f9667640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f4147d60 con 0x7f04e00039b0 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f40cc540 0x7f04f40cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f40cc540 0x7f04f40cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35326/0 (socket says 172.21.15.64:35326) 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 -- 172.21.15.64:0/4037693520 learned_addr learned my addr 172.21.15.64:0/4037693520 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04e00039b0 0x7f04f40b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f37fe640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04f4135320 0x7f04f4161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 -- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04e00039b0 msgr2=0x7f04f40b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04e00039b0 0x7f04f40b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 -- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04f4135320 msgr2=0x7f04f4161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04f4135320 0x7f04f4161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 -- 172.21.15.64:0/4037693520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40cf250 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.505+0000 7f04f27fc640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f40cc540 0x7f04f40cea20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f04dc011700 tx=0x7f04dc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.506+0000 7f04eb7fe640 1 -- 172.21.15.64:0/4037693520 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc015870 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.506+0000 7f04eb7fe640 1 -- 172.21.15.64:0/4037693520 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc015a10 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.506+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f40ccae0 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.506+0000 7f04eb7fe640 1 -- 172.21.15.64:0/4037693520 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc015dc0 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.506+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f40cd0a0 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.507+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f4062c90 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.507+0000 7f04eb7fe640 1 -- 172.21.15.64:0/4037693520 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f04dc00f480 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.507+0000 7f04eb7fe640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f04d406b4d0 0x7f04d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.507+0000 7f04eb7fe640 1 -- 172.21.15.64:0/4037693520 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04dc08a5e0 con 0x7f04f40cc540 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.508+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f04d406b4d0 0x7f04d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.508+0000 7f04f2ffd640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f04d406b4d0 0x7f04d406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f04e4036000 tx=0x7f04e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.512+0000 7f04eb7fe640 1 -- 172.21.15.64:0/4037693520 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f04dc019030 con 0x7f04f40cc540 2024-03-07T07:20:04.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.695+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04f4162f00 con 0x7f04f40cc540 2024-03-07T07:20:04.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.696+0000 7f04eb7fe640 1 -- 172.21.15.64:0/4037693520 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12855 (secure 0 0 0) 0x7f04dc05f090 con 0x7f04f40cc540 2024-03-07T07:20:04.696 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:04.696 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":29,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:20:01.721360+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.697+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f04d406b4d0 msgr2=0x7f04d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.697+0000 7f04f9667640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f04d406b4d0 0x7f04d406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f04e4036000 tx=0x7f04e402e000 comp rx=0 tx=0).stop 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.697+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f40cc540 msgr2=0x7f04f40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.697+0000 7f04f9667640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f40cc540 0x7f04f40cea20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f04dc011700 tx=0x7f04dc011b00 comp rx=0 tx=0).stop 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 shutdown_connections 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f04d406b4d0 0x7f04d406d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04f40cc540 0x7f04f40cea20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04f4135320 0x7f04f4161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 --2- 172.21.15.64:0/4037693520 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04e00039b0 0x7f04f40b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 >> 172.21.15.64:0/4037693520 conn(0x7f04f406d4c0 msgr2=0x7f04f41465a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:04.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 shutdown_connections 2024-03-07T07:20:04.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.698+0000 7f04f9667640 1 -- 172.21.15.64:0/4037693520 wait complete. 2024-03-07T07:20:04.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:20:04.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:20:04.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.812+0000 7f946c51d640 1 Processor -- start 2024-03-07T07:20:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.812+0000 7f946c51d640 1 -- start start 2024-03-07T07:20:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.813+0000 7f946c51d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f94641625c0 0x7f9464071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.813+0000 7f946c51d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94640b7200 0x7f94640717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.813+0000 7f946c51d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464071ff0 con 0x7f946415e350 2024-03-07T07:20:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.813+0000 7f946c51d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94640d2160 con 0x7f94641625c0 2024-03-07T07:20:04.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.813+0000 7f946c51d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94640d23b0 con 0x7f94640b7200 2024-03-07T07:20:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f9469ab8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f94641625c0 0x7f9464071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f946aaba640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94640b7200 0x7f94640717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f946aaba640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94640b7200 0x7f94640717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:37734/0 (socket says 172.21.15.64:37734) 2024-03-07T07:20:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f946aaba640 1 -- 172.21.15.64:0/3963559886 learned_addr learned my addr 172.21.15.64:0/3963559886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f9469ab8640 1 -- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94640b7200 msgr2=0x7f94640717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f9469ab8640 1 --2- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94640b7200 0x7f94640717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f9469ab8640 1 -- 172.21.15.64:0/3963559886 >> v1:172.21.15.120:6789/0 conn(0x7f946415e350 legacy=0x7f9464161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f9469ab8640 1 -- 172.21.15.64:0/3963559886 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640d2690 con 0x7f94641625c0 2024-03-07T07:20:04.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.814+0000 7f946aaba640 1 --2- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94640b7200 0x7f94640717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.815+0000 7f9469ab8640 1 --2- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94641625c0 0x7f9464071280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94540097a0 tx=0x7f945402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a62c2820a150be2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.815+0000 7f94692b7640 1 -- 172.21.15.64:0/3963559886 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454035020 con 0x7f94641625c0 2024-03-07T07:20:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.815+0000 7f94692b7640 1 -- 172.21.15.64:0/3963559886 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454031810 con 0x7f94641625c0 2024-03-07T07:20:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.815+0000 7f94692b7640 1 -- 172.21.15.64:0/3963559886 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454031be0 con 0x7f94641625c0 2024-03-07T07:20:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 -- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94641625c0 msgr2=0x7f9464071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 --2- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94641625c0 0x7f9464071280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94540097a0 tx=0x7f945402b8b0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 -- 172.21.15.64:0/3963559886 shutdown_connections 2024-03-07T07:20:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 --2- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94640b7200 0x7f94640717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 --2- 172.21.15.64:0/3963559886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94641625c0 0x7f9464071280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 -- 172.21.15.64:0/3963559886 >> 172.21.15.64:0/3963559886 conn(0x7f94640d0e70 msgr2=0x7f94640d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 -- 172.21.15.64:0/3963559886 shutdown_connections 2024-03-07T07:20:04.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.816+0000 7f946c51d640 1 -- 172.21.15.64:0/3963559886 wait complete. 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.817+0000 7f946c51d640 1 Processor -- start 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.817+0000 7f946c51d640 1 -- start start 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946c51d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94640b7200 0x7f946415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946c51d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f946415e350 0x7f94641e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946c51d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94641625c0 0x7f94641ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946c51d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946415e730 con 0x7f94640b7200 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946c51d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9464071d00 con 0x7f946415e350 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946c51d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94640d2e90 con 0x7f94641625c0 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94640b7200 0x7f946415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94640b7200 0x7f946415cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35338/0 (socket says 172.21.15.64:35338) 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 -- 172.21.15.64:0/3555151871 learned_addr learned my addr 172.21.15.64:0/3555151871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f9469ab8640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f946415e350 0x7f94641e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946aaba640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94641625c0 0x7f94641ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 -- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94641625c0 msgr2=0x7f94641ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94641625c0 0x7f94641ea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 -- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f946415e350 msgr2=0x7f94641e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f946415e350 0x7f94641e9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 -- 172.21.15.64:0/3555151871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94641e8950 con 0x7f94640b7200 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946a2b9640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94640b7200 0x7f946415cab0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f946000c400 tx=0x7f946000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f945affd640 1 -- 172.21.15.64:0/3555151871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600052d0 con 0x7f94640b7200 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f945affd640 1 -- 172.21.15.64:0/3555151871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9460005490 con 0x7f94640b7200 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94641e8c30 con 0x7f94640b7200 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.818+0000 7f945affd640 1 -- 172.21.15.64:0/3555151871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460007260 con 0x7f94640b7200 2024-03-07T07:20:04.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.819+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94641e91f0 con 0x7f94640b7200 2024-03-07T07:20:04.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.820+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9464062c90 con 0x7f94640b7200 2024-03-07T07:20:04.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.820+0000 7f945affd640 1 -- 172.21.15.64:0/3555151871 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9460007400 con 0x7f94640b7200 2024-03-07T07:20:04.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.820+0000 7f945affd640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f944806b4d0 0x7f944806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:04.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.820+0000 7f945affd640 1 -- 172.21.15.64:0/3555151871 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94600894b0 con 0x7f94640b7200 2024-03-07T07:20:04.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.821+0000 7f9469ab8640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f944806b4d0 0x7f944806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:04.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.821+0000 7f9469ab8640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f944806b4d0 0x7f944806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9454036000 tx=0x7f945403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:04.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:04.825+0000 7f945affd640 1 -- 172.21.15.64:0/3555151871 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9460058600 con 0x7f94640b7200 2024-03-07T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.005+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94640b3780 con 0x7f94640b7200 2024-03-07T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.005+0000 7f945affd640 1 -- 172.21.15.64:0/3555151871 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94640b3780 con 0x7f94640b7200 2024-03-07T07:20:05.005 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f944806b4d0 msgr2=0x7f944806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f944806b4d0 0x7f944806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9454036000 tx=0x7f945403c040 comp rx=0 tx=0).stop 2024-03-07T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94640b7200 msgr2=0x7f946415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94640b7200 0x7f946415cab0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f946000c400 tx=0x7f946000c8c0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 shutdown_connections 2024-03-07T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f944806b4d0 0x7f944806d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f94641625c0 0x7f94641ea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f946415e350 0x7f94641e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 --2- 172.21.15.64:0/3555151871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94640b7200 0x7f946415cab0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 >> 172.21.15.64:0/3555151871 conn(0x7f94640d0e70 msgr2=0x7f94640b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 shutdown_connections 2024-03-07T07:20:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.007+0000 7f946c51d640 1 -- 172.21.15.64:0/3555151871 wait complete. 2024-03-07T07:20:05.018 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-07T07:20:05.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.119+0000 7f00088dd640 1 Processor -- start 2024-03-07T07:20:05.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.119+0000 7f00088dd640 1 -- start start 2024-03-07T07:20:05.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.119+0000 7f00088dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f000415e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f00088dd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0004071c50 con 0x7f00040d10a0 2024-03-07T07:20:05.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f00088dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0004071f30 con 0x7f0004072cf0 2024-03-07T07:20:05.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f00088dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000406c3a0 con 0x7f000415f170 2024-03-07T07:20:05.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f0002d9d640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f00040d10a0 0x7f00041470e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56464/0 (socket says 172.21.15.64:56464) 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f0002d9d640 1 -- 172.21.15.64:0/2315508195 learned_addr learned my addr 172.21.15.64:0/2315508195 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f000259c640 1 --2- 172.21.15.64:0/2315508195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f000415e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128743895 0 0) 0x7f0004071c50 con 0x7f00040d10a0 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff0003520 con 0x7f00040d10a0 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979019728 0 0) 0x7f0004071f30 con 0x7f0004072cf0 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0004071c50 con 0x7f0004072cf0 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099140680 0 0) 0x7efff0003520 con 0x7f00040d10a0 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f000259c640 1 -- 172.21.15.64:0/2315508195 >> v1:172.21.15.120:6789/0 conn(0x7f0004072cf0 legacy=0x7f00040d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f000259c640 1 -- 172.21.15.64:0/2315508195 >> v1:172.21.15.64:6789/0 conn(0x7f00040d10a0 legacy=0x7f00041470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.120+0000 7f000259c640 1 -- 172.21.15.64:0/2315508195 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004072210 con 0x7f000415f170 2024-03-07T07:20:05.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.121+0000 7f000259c640 1 --2- 172.21.15.64:0/2315508195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f000415e650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efff40097d0 tx=0x7efff402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b1816fc68d794b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:05.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.121+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4037070 con 0x7f000415f170 2024-03-07T07:20:05.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.121+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4030510 con 0x7f000415f170 2024-03-07T07:20:05.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.121+0000 7f0001d9b640 1 -- 172.21.15.64:0/2315508195 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc0019f0 con 0x7f000415f170 2024-03-07T07:20:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.122+0000 7f00088dd640 1 -- 172.21.15.64:0/2315508195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 msgr2=0x7f000415e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.122+0000 7f00088dd640 1 --2- 172.21.15.64:0/2315508195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f000415e650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efff40097d0 tx=0x7efff402b800 comp rx=0 tx=0).stop 2024-03-07T07:20:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.122+0000 7f00088dd640 1 -- 172.21.15.64:0/2315508195 shutdown_connections 2024-03-07T07:20:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.122+0000 7f00088dd640 1 --2- 172.21.15.64:0/2315508195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f000415e650 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.122+0000 7f00088dd640 1 -- 172.21.15.64:0/2315508195 >> 172.21.15.64:0/2315508195 conn(0x7f000406b480 msgr2=0x7f000406b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.122+0000 7f00088dd640 1 -- 172.21.15.64:0/2315508195 shutdown_connections 2024-03-07T07:20:05.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.122+0000 7f00088dd640 1 -- 172.21.15.64:0/2315508195 wait complete. 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.123+0000 7f00088dd640 1 Processor -- start 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.123+0000 7f00088dd640 1 -- start start 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f00088dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0004072cf0 0x7f00041435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f00088dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00040d10a0 0x7f000413f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f00088dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f00041f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f00088dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004160560 con 0x7f00040d10a0 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f00088dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040727d0 con 0x7f0004072cf0 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f00088dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0004071550 con 0x7f000415f170 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00040d10a0 0x7f000413f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00040d10a0 0x7f000413f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35354/0 (socket says 172.21.15.64:35354) 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 -- 172.21.15.64:0/1437962050 learned_addr learned my addr 172.21.15.64:0/1437962050 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f000259c640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0004072cf0 0x7f00041435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7f0002d9d640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f00041f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 -- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 msgr2=0x7f00041f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f00041f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 -- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0004072cf0 msgr2=0x7f00041435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0004072cf0 0x7f00041435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 -- 172.21.15.64:0/1437962050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000413f810 con 0x7f00040d10a0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.124+0000 7efffbfff640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00040d10a0 0x7f000413f2d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7effec00c450 tx=0x7effec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.125+0000 7efffaffd640 1 -- 172.21.15.64:0/1437962050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0052d0 con 0x7f00040d10a0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.125+0000 7efffaffd640 1 -- 172.21.15.64:0/1437962050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec005470 con 0x7f00040d10a0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.125+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00040cb4c0 con 0x7f00040d10a0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.125+0000 7efffaffd640 1 -- 172.21.15.64:0/1437962050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec012870 con 0x7f00040d10a0 2024-03-07T07:20:05.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.125+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00041f7d00 con 0x7f00040d10a0 2024-03-07T07:20:05.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.126+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0004062c90 con 0x7f00040d10a0 2024-03-07T07:20:05.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.126+0000 7efffaffd640 1 -- 172.21.15.64:0/1437962050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7effec012a10 con 0x7f00040d10a0 2024-03-07T07:20:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.126+0000 7efffaffd640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7effe406b4d0 0x7effe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.127+0000 7efffaffd640 1 -- 172.21.15.64:0/1437962050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7effec0899b0 con 0x7f00040d10a0 2024-03-07T07:20:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.127+0000 7f000259c640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7effe406b4d0 0x7effe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.127+0000 7f000259c640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7effe406b4d0 0x7effe406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efff4034000 tx=0x7efff4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:05.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.131+0000 7efffaffd640 1 -- 172.21.15.64:0/1437962050 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7effec058600 con 0x7f00040d10a0 2024-03-07T07:20:05.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.312+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f00040b35b0 con 0x7f00040d10a0 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.313+0000 7efffaffd640 1 -- 172.21.15.64:0/1437962050 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7effec05c6a0 con 0x7f00040d10a0 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-03-07T07:20:05.313 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-03-07T07:20:05.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7effe406b4d0 msgr2=0x7effe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7effe406b4d0 0x7effe406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efff4034000 tx=0x7efff4008030 comp rx=0 tx=0).stop 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00040d10a0 msgr2=0x7f000413f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00040d10a0 0x7f000413f2d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7effec00c450 tx=0x7effec00c910 comp rx=0 tx=0).stop 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 shutdown_connections 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7effe406b4d0 0x7effe406d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00040d10a0 0x7f000413f2d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f000415f170 0x7f00041f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 --2- 172.21.15.64:0/1437962050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0004072cf0 0x7f00041435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 >> 172.21.15.64:0/1437962050 conn(0x7f000406b480 msgr2=0x7f000415f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 shutdown_connections 2024-03-07T07:20:05.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.314+0000 7f00088dd640 1 -- 172.21.15.64:0/1437962050 wait complete. 2024-03-07T07:20:05.325 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-07T07:20:05.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-07T07:20:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.430+0000 7f751a761640 1 Processor -- start 2024-03-07T07:20:05.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.430+0000 7f751a761640 1 -- start start 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f751a761640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7514141240 0x7f75140b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f751a761640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f75140b4f50 0x7f75140b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f751a761640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7514141620 con 0x7f75140b4040 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f751a761640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75140b4490 con 0x7f75140b4f50 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f751a761640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7514162970 con 0x7f7514141240 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f75137fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f75140b4f50 0x7f75140b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f7513fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7514141240 0x7f75140b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f7518cfe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f75140b4040 0x7f75141438a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:53800/0 (socket says 172.21.15.64:53800) 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f7518cfe640 1 -- 172.21.15.64:0/1279483171 learned_addr learned my addr 172.21.15.64:0/1279483171 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f75137fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f75140b4f50 0x7f75140b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:59858/0 (socket says 172.21.15.64:59858) 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f75137fe640 1 -- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7514141240 msgr2=0x7f75140b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f75137fe640 1 --2- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7514141240 0x7f75140b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f75137fe640 1 -- 172.21.15.64:0/1279483171 >> v1:172.21.15.120:6789/0 conn(0x7f75140b4040 legacy=0x7f75141438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f75137fe640 1 -- 172.21.15.64:0/1279483171 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514161da0 con 0x7f75140b4f50 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f7512ffd640 1 -- 172.21.15.64:0/1279483171 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246975333 0 0) 0x7f750c001490 con 0x7f75140b4040 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.431+0000 7f7513fff640 1 --2- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7514141240 0x7f75140b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.432+0000 7f75137fe640 1 --2- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75140b4f50 0x7f75140b3b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75040097a0 tx=0x7f750402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82608e8098d743ec server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.432+0000 7f7512ffd640 1 -- 172.21.15.64:0/1279483171 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504035020 con 0x7f75140b4f50 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.432+0000 7f7512ffd640 1 -- 172.21.15.64:0/1279483171 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504031830 con 0x7f75140b4f50 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.432+0000 7f7512ffd640 1 -- 172.21.15.64:0/1279483171 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504031c00 con 0x7f75140b4f50 2024-03-07T07:20:05.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 -- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75140b4f50 msgr2=0x7f75140b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 --2- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75140b4f50 0x7f75140b3b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75040097a0 tx=0x7f750402b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 -- 172.21.15.64:0/1279483171 shutdown_connections 2024-03-07T07:20:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 --2- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f75140b4f50 0x7f75140b3b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 --2- 172.21.15.64:0/1279483171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7514141240 0x7f75140b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 -- 172.21.15.64:0/1279483171 >> 172.21.15.64:0/1279483171 conn(0x7f751406d970 msgr2=0x7f7514066eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 -- 172.21.15.64:0/1279483171 shutdown_connections 2024-03-07T07:20:05.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.433+0000 7f751a761640 1 -- 172.21.15.64:0/1279483171 wait complete. 2024-03-07T07:20:05.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.434+0000 7f751a761640 1 Processor -- start 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.434+0000 7f751a761640 1 -- start start 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f751a761640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75140b4040 0x7f7514158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f751a761640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75140b4f50 0x7f75141f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f751a761640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7514141240 0x7f75140b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f751a761640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514162600 con 0x7f75140b4f50 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f751a761640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514162320 con 0x7f7514141240 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f751a761640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7514141900 con 0x7f75140b4040 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75140b4f50 0x7f75141f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75140b4f50 0x7f75141f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35362/0 (socket says 172.21.15.64:35362) 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 -- 172.21.15.64:0/2384833370 learned_addr learned my addr 172.21.15.64:0/2384833370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f7518cfe640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7514141240 0x7f75140b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f7513fff640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75140b4040 0x7f7514158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 -- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75140b4040 msgr2=0x7f7514158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75140b4040 0x7f7514158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 -- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7514141240 msgr2=0x7f75140b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7514141240 0x7f75140b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.435+0000 7f75137fe640 1 -- 172.21.15.64:0/2384833370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751415ad80 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.436+0000 7f75137fe640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75140b4f50 0x7f75141f4e60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75040097a0 tx=0x7f75040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.436+0000 7f7510ff9640 1 -- 172.21.15.64:0/2384833370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504035070 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.436+0000 7f7510ff9640 1 -- 172.21.15.64:0/2384833370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504004230 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.436+0000 7f7510ff9640 1 -- 172.21.15.64:0/2384833370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750403a630 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.436+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751415b060 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.436+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751415b620 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.437+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7514062c90 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.438+0000 7f7510ff9640 1 -- 172.21.15.64:0/2384833370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7504038020 con 0x7f75140b4f50 2024-03-07T07:20:05.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.438+0000 7f7510ff9640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74f406b4d0 0x7f74f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.438+0000 7f7510ff9640 1 -- 172.21.15.64:0/2384833370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f75040aba90 con 0x7f75140b4f50 2024-03-07T07:20:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.438+0000 7f7513fff640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74f406b4d0 0x7f74f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:05.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.439+0000 7f7513fff640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74f406b4d0 0x7f74f406d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009a30 tx=0x7f74fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:05.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.442+0000 7f7510ff9640 1 -- 172.21.15.64:0/2384833370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7504040070 con 0x7f75140b4f50 2024-03-07T07:20:05.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:05.620+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f751406c0a0 con 0x7f75140b4f50 2024-03-07T07:20:06.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.004+0000 7f7510ff9640 1 -- 172.21.15.64:0/2384833370 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f750407acf0 con 0x7f75140b4f50 2024-03-07T07:20:06.004 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74f406b4d0 msgr2=0x7f74f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74f406b4d0 0x7f74f406d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009a30 tx=0x7f74fc008040 comp rx=0 tx=0).stop 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75140b4f50 msgr2=0x7f75141f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75140b4f50 0x7f75141f4e60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75040097a0 tx=0x7f75040046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 shutdown_connections 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74f406b4d0 0x7f74f406d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75140b4f50 0x7f75141f4e60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7514141240 0x7f75140b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 --2- 172.21.15.64:0/2384833370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f75140b4040 0x7f7514158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 >> 172.21.15.64:0/2384833370 conn(0x7f751406d970 msgr2=0x7f75141407c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 shutdown_connections 2024-03-07T07:20:06.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.006+0000 7f751a761640 1 -- 172.21.15.64:0/2384833370 wait complete. 2024-03-07T07:20:06.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-07T07:20:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.131+0000 7fa83222b640 1 Processor -- start 2024-03-07T07:20:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.132+0000 7fa83222b640 1 -- start start 2024-03-07T07:20:06.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.132+0000 7fa83222b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa82c161e10 0x7fa82c144450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.132+0000 7fa83222b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c15e820 con 0x7fa82c141170 2024-03-07T07:20:06.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.132+0000 7fa83222b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0b9810 con 0x7fa82c0b5610 2024-03-07T07:20:06.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.132+0000 7fa83222b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c144990 con 0x7fa82c161e10 2024-03-07T07:20:06.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.132+0000 7fa82b7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa82c141170 0x7fa82c1419b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56468/0 (socket says 172.21.15.64:56468) 2024-03-07T07:20:06.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.132+0000 7fa82b7fe640 1 -- 172.21.15.64:0/3106512024 learned_addr learned my addr 172.21.15.64:0/3106512024 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436373781 0 0) 0x7fa81c001110 con 0x7fa82c141170 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa814003520 con 0x7fa82c141170 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82affd640 1 --2- 172.21.15.64:0/3106512024 >> v2:172.21.15.120:3300/0 conn(0x7fa82c161e10 0x7fa82c144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842245689 0 0) 0x7fa814003520 con 0x7fa82c141170 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 >> v2:172.21.15.120:3300/0 conn(0x7fa82c161e10 msgr2=0x7fa82c144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 --2- 172.21.15.64:0/3106512024 >> v2:172.21.15.120:3300/0 conn(0x7fa82c161e10 0x7fa82c144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 >> v1:172.21.15.120:6789/0 conn(0x7fa82c0b5610 legacy=0x7fa82c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c162310 con 0x7fa82c141170 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7fa81c001840 con 0x7fa82c141170 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 >> v1:172.21.15.64:6789/0 conn(0x7fa82c141170 legacy=0x7fa82c1419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8140039b0 0x7fa814003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa814004800 0x7fa814004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8140051c0 0x7fa81400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c144990 con 0x7fa814004800 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81400c060 con 0x7fa8140051c0 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81400c260 con 0x7fa8140039b0 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82affd640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa814004800 0x7fa814004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82b7fe640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8140039b0 0x7fa814003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82bfff640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8140051c0 0x7fa81400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82affd640 1 -- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8140039b0 msgr2=0x7fa814003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82affd640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8140039b0 0x7fa814003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82affd640 1 -- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8140051c0 msgr2=0x7fa81400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82affd640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8140051c0 0x7fa81400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82affd640 1 -- 172.21.15.64:0/3106512024 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814003520 con 0x7fa814004800 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.133+0000 7fa82bfff640 1 -- 172.21.15.64:0/3106512024 reap_dead start 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.134+0000 7fa82affd640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa814004800 0x7fa814004c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa820009a90 tx=0x7fa814005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b919b2b3a7e28f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:06.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.134+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820039070 con 0x7fa814004800 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.134+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82002c680 con 0x7fa814004800 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.134+0000 7fa82a7fc640 1 -- 172.21.15.64:0/3106512024 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820009690 con 0x7fa814004800 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.135+0000 7fa83222b640 1 -- 172.21.15.64:0/3106512024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa814004800 msgr2=0x7fa814004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.135+0000 7fa83222b640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa814004800 0x7fa814004c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa820009a90 tx=0x7fa814005830 comp rx=0 tx=0).stop 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.135+0000 7fa83222b640 1 -- 172.21.15.64:0/3106512024 shutdown_connections 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.135+0000 7fa83222b640 1 --2- 172.21.15.64:0/3106512024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa814004800 0x7fa814004c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.135+0000 7fa83222b640 1 -- 172.21.15.64:0/3106512024 >> 172.21.15.64:0/3106512024 conn(0x7fa82c0d2270 msgr2=0x7fa82c0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.135+0000 7fa83222b640 1 -- 172.21.15.64:0/3106512024 shutdown_connections 2024-03-07T07:20:06.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.135+0000 7fa83222b640 1 -- 172.21.15.64:0/3106512024 wait complete. 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.136+0000 7fa83222b640 1 Processor -- start 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.136+0000 7fa83222b640 1 -- start start 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa83222b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa814004800 0x7fa82c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa83222b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa82c071740 0x7fa82c071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa83222b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa82c15db00 0x7fa82c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa83222b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c162bb0 con 0x7fa82c15db00 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa83222b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1628d0 con 0x7fa82c071740 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa83222b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c141550 con 0x7fa814004800 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa82c15db00 0x7fa82c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa82c15db00 0x7fa82c15df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35384/0 (socket says 172.21.15.64:35384) 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 -- 172.21.15.64:0/1090980835 learned_addr learned my addr 172.21.15.64:0/1090980835 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:06.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82affd640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa82c071740 0x7fa82c071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82b7fe640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa814004800 0x7fa82c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 -- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa814004800 msgr2=0x7fa82c157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa814004800 0x7fa82c157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 -- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa82c071740 msgr2=0x7fa82c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa82c071740 0x7fa82c071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82bfff640 1 -- 172.21.15.64:0/1090980835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c1f7570 con 0x7fa82c15db00 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.137+0000 7fa82b7fe640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa814004800 0x7fa82c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.138+0000 7fa82bfff640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa82c15db00 0x7fa82c15df80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa824007fd0 tx=0x7fa82400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.138+0000 7fa813fff640 1 -- 172.21.15.64:0/1090980835 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824019070 con 0x7fa82c15db00 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.138+0000 7fa813fff640 1 -- 172.21.15.64:0/1090980835 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa824006d60 con 0x7fa82c15db00 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.138+0000 7fa813fff640 1 -- 172.21.15.64:0/1090980835 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824003b90 con 0x7fa82c15db00 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.138+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c1f7850 con 0x7fa82c15db00 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.138+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c1f7e10 con 0x7fa82c15db00 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.139+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c062c90 con 0x7fa82c15db00 2024-03-07T07:20:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.139+0000 7fa813fff640 1 -- 172.21.15.64:0/1090980835 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa82401c070 con 0x7fa82c15db00 2024-03-07T07:20:06.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.140+0000 7fa813fff640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa81806b500 0x7fa81806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:06.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.140+0000 7fa813fff640 1 -- 172.21.15.64:0/1090980835 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa824089440 con 0x7fa82c15db00 2024-03-07T07:20:06.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.140+0000 7fa82b7fe640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa81806b500 0x7fa81806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:06.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.140+0000 7fa82b7fe640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa81806b500 0x7fa81806d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0013d0 tx=0x7fa81c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:06.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.144+0000 7fa813fff640 1 -- 172.21.15.64:0/1090980835 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa824057e50 con 0x7fa82c15db00 2024-03-07T07:20:06.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:06.325+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa82c0d3e00 con 0x7fa82c15db00 2024-03-07T07:20:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.009+0000 7fa813fff640 1 -- 172.21.15.64:0/1090980835 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa82405bef0 con 0x7fa82c15db00 2024-03-07T07:20:07.009 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-03-07T07:20:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa81806b500 msgr2=0x7fa81806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa81806b500 0x7fa81806d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0013d0 tx=0x7fa81c009040 comp rx=0 tx=0).stop 2024-03-07T07:20:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa82c15db00 msgr2=0x7fa82c15df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa82c15db00 0x7fa82c15df80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa824007fd0 tx=0x7fa82400e960 comp rx=0 tx=0).stop 2024-03-07T07:20:07.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 shutdown_connections 2024-03-07T07:20:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa81806b500 0x7fa81806d9c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa82c15db00 0x7fa82c15df80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa82c071740 0x7fa82c071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 --2- 172.21.15.64:0/1090980835 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa814004800 0x7fa82c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 >> 172.21.15.64:0/1090980835 conn(0x7fa82c0d2270 msgr2=0x7fa82c143860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 shutdown_connections 2024-03-07T07:20:07.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.012+0000 7fa83222b640 1 -- 172.21.15.64:0/1090980835 wait complete. 2024-03-07T07:20:07.035 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-07T07:20:07.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.141+0000 7fb617b1d640 1 Processor -- start 2024-03-07T07:20:07.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb617b1d640 1 -- start start 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb617b1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb610141360 0x7fb610141910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb617b1d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb6100b44f0 0x7fb6101620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb617b1d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61015ebd0 con 0x7fb6100b5720 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb617b1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61015eeb0 con 0x7fb610141360 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb617b1d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61015f190 con 0x7fb6100b44f0 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb6158b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb610141360 0x7fb610141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.142+0000 7fb6158b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fb610141360 0x7fb610141910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35386/0 (socket says 172.21.15.64:35386) 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.143+0000 7fb6158b9640 1 -- 172.21.15.64:0/1669566087 learned_addr learned my addr 172.21.15.64:0/1669566087 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.143+0000 7fb6160ba640 1 --2- 172.21.15.64:0/1669566087 >> v2:172.21.15.120:3300/0 conn(0x7fb6100b44f0 0x7fb6101620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:07.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.143+0000 7fb6158b9640 1 -- 172.21.15.64:0/1669566087 >> v2:172.21.15.120:3300/0 conn(0x7fb6100b44f0 msgr2=0x7fb6101620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:07.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.143+0000 7fb6158b9640 1 --2- 172.21.15.64:0/1669566087 >> v2:172.21.15.120:3300/0 conn(0x7fb6100b44f0 0x7fb6101620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.143+0000 7fb6158b9640 1 -- 172.21.15.64:0/1669566087 >> v1:172.21.15.120:6789/0 conn(0x7fb6100b5720 legacy=0x7fb6100b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:07.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.143+0000 7fb6158b9640 1 -- 172.21.15.64:0/1669566087 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61015f470 con 0x7fb610141360 2024-03-07T07:20:07.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.143+0000 7fb6160ba640 1 --2- 172.21.15.64:0/1669566087 >> v2:172.21.15.120:3300/0 conn(0x7fb6100b44f0 0x7fb6101620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:07.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb6158b9640 1 --2- 172.21.15.64:0/1669566087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb610141910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b4af0 tx=0x7fb60402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451aa5209c377c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:07.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb6148b7640 1 -- 172.21.15.64:0/1669566087 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604034070 con 0x7fb610141360 2024-03-07T07:20:07.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb6148b7640 1 -- 172.21.15.64:0/1669566087 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604030860 con 0x7fb610141360 2024-03-07T07:20:07.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb6148b7640 1 -- 172.21.15.64:0/1669566087 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604030c30 con 0x7fb610141360 2024-03-07T07:20:07.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 -- 172.21.15.64:0/1669566087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 msgr2=0x7fb610141910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1669566087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb610141910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b4af0 tx=0x7fb60402b9a0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 -- 172.21.15.64:0/1669566087 shutdown_connections 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1669566087 >> v2:172.21.15.120:3300/0 conn(0x7fb6100b44f0 0x7fb6101620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1669566087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb610141910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 -- 172.21.15.64:0/1669566087 >> 172.21.15.64:0/1669566087 conn(0x7fb6100705e0 msgr2=0x7fb6100709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 -- 172.21.15.64:0/1669566087 shutdown_connections 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.144+0000 7fb617b1d640 1 -- 172.21.15.64:0/1669566087 wait complete. 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 Processor -- start 2024-03-07T07:20:07.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 -- start start 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6100b44f0 0x7fb6101f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6100b5720 0x7fb6101e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb6100725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100b5bd0 con 0x7fb610141360 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61015e8c0 con 0x7fb6100b44f0 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb617b1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100b5d50 con 0x7fb6100b5720 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb6160ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb6100725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb6160ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb6100725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35402/0 (socket says 172.21.15.64:35402) 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.146+0000 7fb6160ba640 1 -- 172.21.15.64:0/1106085869 learned_addr learned my addr 172.21.15.64:0/1106085869 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6158b9640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6100b44f0 0x7fb6101f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:07.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6150b8640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6100b5720 0x7fb6101e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6160ba640 1 -- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6100b5720 msgr2=0x7fb6101e76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6160ba640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6100b5720 0x7fb6101e76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6160ba640 1 -- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6100b44f0 msgr2=0x7fb6101f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6160ba640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6100b44f0 0x7fb6101f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6160ba640 1 -- 172.21.15.64:0/1106085869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610159010 con 0x7fb610141360 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6158b9640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6100b44f0 0x7fb6101f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6150b8640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6100b5720 0x7fb6101e76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6160ba640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb6100725a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb608009fd0 tx=0x7fb60800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6027fc640 1 -- 172.21.15.64:0/1106085869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60801a070 con 0x7fb610141360 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6027fc640 1 -- 172.21.15.64:0/1106085869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608003bb0 con 0x7fb610141360 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb6027fc640 1 -- 172.21.15.64:0/1106085869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608004030 con 0x7fb610141360 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6101592f0 con 0x7fb610141360 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.147+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6101e7c20 con 0x7fb610141360 2024-03-07T07:20:07.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.148+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb610062c90 con 0x7fb610141360 2024-03-07T07:20:07.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.148+0000 7fb6027fc640 1 -- 172.21.15.64:0/1106085869 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb60801d070 con 0x7fb610141360 2024-03-07T07:20:07.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.149+0000 7fb6027fc640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb5f406b4d0 0x7fb5f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:07.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.149+0000 7fb6027fc640 1 -- 172.21.15.64:0/1106085869 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb60808a490 con 0x7fb610141360 2024-03-07T07:20:07.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.149+0000 7fb6158b9640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb5f406b4d0 0x7fb5f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:07.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.150+0000 7fb6158b9640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb5f406b4d0 0x7fb5f406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb604036000 tx=0x7fb60403b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:07.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.154+0000 7fb6027fc640 1 -- 172.21.15.64:0/1106085869 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb608010070 con 0x7fb610141360 2024-03-07T07:20:07.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:07.374+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb6100d29f0 con 0x7fb610141360 2024-03-07T07:20:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.011+0000 7fb6027fc640 1 -- 172.21.15.64:0/1106085869 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb608059370 con 0x7fb610141360 2024-03-07T07:20:08.011 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 4 and data pool 5 2024-03-07T07:20:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb5f406b4d0 msgr2=0x7fb5f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb5f406b4d0 0x7fb5f406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb604036000 tx=0x7fb60403b040 comp rx=0 tx=0).stop 2024-03-07T07:20:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 msgr2=0x7fb6100725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb6100725a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb608009fd0 tx=0x7fb60800d560 comp rx=0 tx=0).stop 2024-03-07T07:20:08.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 shutdown_connections 2024-03-07T07:20:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb5f406b4d0 0x7fb5f406d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb610141360 0x7fb6100725a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6100b5720 0x7fb6101e76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 --2- 172.21.15.64:0/1106085869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb6100b44f0 0x7fb6101f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.014+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 >> 172.21.15.64:0/1106085869 conn(0x7fb6100705e0 msgr2=0x7fb610160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:08.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.019+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 shutdown_connections 2024-03-07T07:20:08.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.019+0000 7fb617b1d640 1 -- 172.21.15.64:0/1106085869 wait complete. 2024-03-07T07:20:08.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.152+0000 7fe5ee634640 1 Processor -- start 2024-03-07T07:20:08.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.152+0000 7fe5ee634640 1 -- start start 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5ee634640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5e80b55c0 0x7fe5e80b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5ee634640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e8147530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5ee634640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e80b4520 con 0x7fe5e815e0d0 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5ee634640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e80b5f60 con 0x7fe5e80b55c0 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5ee634640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80b4800 con 0x7fe5e8147080 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5e77fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5e80b55c0 0x7fe5e80b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5ecbd1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e8147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.153+0000 7fe5e7fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe5e815e0d0 0x7fe5e81417d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:53826/0 (socket says 172.21.15.64:53826) 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5e77fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5e80b55c0 0x7fe5e80b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:59926/0 (socket says 172.21.15.64:59926) 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5e7fff640 1 -- 172.21.15.64:0/4040073211 learned_addr learned my addr 172.21.15.64:0/4040073211 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5ecbd1640 1 -- 172.21.15.64:0/4040073211 >> v2:172.21.15.120:3300/0 conn(0x7fe5e80b55c0 msgr2=0x7fe5e80b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:08.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5ecbd1640 1 --2- 172.21.15.64:0/4040073211 >> v2:172.21.15.120:3300/0 conn(0x7fe5e80b55c0 0x7fe5e80b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5ecbd1640 1 -- 172.21.15.64:0/4040073211 >> v1:172.21.15.120:6789/0 conn(0x7fe5e815e0d0 legacy=0x7fe5e81417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5ecbd1640 1 -- 172.21.15.64:0/4040073211 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80b4ae0 con 0x7fe5e8147080 2024-03-07T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5e6ffd640 1 -- 172.21.15.64:0/4040073211 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585855655 0 0) 0x7fe5e80b4520 con 0x7fe5e815e0d0 2024-03-07T07:20:08.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.154+0000 7fe5e77fe640 1 --2- 172.21.15.64:0/4040073211 >> v2:172.21.15.120:3300/0 conn(0x7fe5e80b55c0 0x7fe5e80b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ecbd1640 1 --2- 172.21.15.64:0/4040073211 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e8147530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009d90 tx=0x7fe5e002bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ebc37e7130f7a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5e6ffd640 1 -- 172.21.15.64:0/4040073211 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0039070 con 0x7fe5e8147080 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5e6ffd640 1 -- 172.21.15.64:0/4040073211 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e002cb30 con 0x7fe5e8147080 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5e6ffd640 1 -- 172.21.15.64:0/4040073211 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0034440 con 0x7fe5e8147080 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 -- 172.21.15.64:0/4040073211 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 msgr2=0x7fe5e8147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 --2- 172.21.15.64:0/4040073211 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e8147530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009d90 tx=0x7fe5e002bd80 comp rx=0 tx=0).stop 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 -- 172.21.15.64:0/4040073211 shutdown_connections 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 --2- 172.21.15.64:0/4040073211 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e8147530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 --2- 172.21.15.64:0/4040073211 >> v2:172.21.15.120:3300/0 conn(0x7fe5e80b55c0 0x7fe5e80b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 -- 172.21.15.64:0/4040073211 >> 172.21.15.64:0/4040073211 conn(0x7fe5e8071300 msgr2=0x7fe5e8071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 -- 172.21.15.64:0/4040073211 shutdown_connections 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.155+0000 7fe5ee634640 1 -- 172.21.15.64:0/4040073211 wait complete. 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 Processor -- start 2024-03-07T07:20:08.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 -- start start 2024-03-07T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e80b55c0 0x7fe5e81f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e80b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e815e0d0 0x7fe5e80b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e8141ee0 con 0x7fe5e815e0d0 2024-03-07T07:20:08.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80b50a0 con 0x7fe5e80b55c0 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.157+0000 7fe5ee634640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e815e4b0 con 0x7fe5e8147080 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e815e0d0 0x7fe5e80b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5e7fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e80b55c0 0x7fe5e81f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e815e0d0 0x7fe5e80b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35404/0 (socket says 172.21.15.64:35404) 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 -- 172.21.15.64:0/2575890120 learned_addr learned my addr 172.21.15.64:0/2575890120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5e77fe640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e80b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 -- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 msgr2=0x7fe5e80b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e80b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 -- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e80b55c0 msgr2=0x7fe5e81f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e80b55c0 0x7fe5e81f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 -- 172.21.15.64:0/2575890120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e815ea50 con 0x7fe5e815e0d0 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ecbd1640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e815e0d0 0x7fe5e80b8650 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009d90 tx=0x7fe5e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5e4ff9640 1 -- 172.21.15.64:0/2575890120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0044020 con 0x7fe5e815e0d0 2024-03-07T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5e4ff9640 1 -- 172.21.15.64:0/2575890120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e003a040 con 0x7fe5e815e0d0 2024-03-07T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5e4ff9640 1 -- 172.21.15.64:0/2575890120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0039050 con 0x7fe5e815e0d0 2024-03-07T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e815ed30 con 0x7fe5e815e0d0 2024-03-07T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.158+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e815f2f0 con 0x7fe5e815e0d0 2024-03-07T07:20:08.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.160+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e8062c90 con 0x7fe5e815e0d0 2024-03-07T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.160+0000 7fe5e4ff9640 1 -- 172.21.15.64:0/2575890120 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe5e0003bb0 con 0x7fe5e815e0d0 2024-03-07T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.160+0000 7fe5e4ff9640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe5d406b4d0 0x7fe5d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.160+0000 7fe5e4ff9640 1 -- 172.21.15.64:0/2575890120 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5e00ac650 con 0x7fe5e815e0d0 2024-03-07T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.160+0000 7fe5e7fff640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe5d406b4d0 0x7fe5d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.161+0000 7fe5e7fff640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe5d406b4d0 0x7fe5d406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc000f60 tx=0x7fe5dc007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.164+0000 7fe5e4ff9640 1 -- 172.21.15.64:0/2575890120 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe5e004e060 con 0x7fe5e815e0d0 2024-03-07T07:20:08.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.349+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e8161a20 con 0x7fe5e815e0d0 2024-03-07T07:20:08.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.350+0000 7fe5e4ff9640 1 -- 172.21.15.64:0/2575890120 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16646 (secure 0 0 0) 0x7fe5e007af10 con 0x7fe5e815e0d0 2024-03-07T07:20:08.350 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:08.350 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:20:07.375395+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:20:05.640176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T07:20:06.344694+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:20:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe5d406b4d0 msgr2=0x7fe5d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe5d406b4d0 0x7fe5d406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc000f60 tx=0x7fe5dc007d50 comp rx=0 tx=0).stop 2024-03-07T07:20:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e815e0d0 msgr2=0x7fe5e80b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e815e0d0 0x7fe5e80b8650 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009d90 tx=0x7fe5e0004a40 comp rx=0 tx=0).stop 2024-03-07T07:20:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 shutdown_connections 2024-03-07T07:20:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe5d406b4d0 0x7fe5d406d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5e815e0d0 0x7fe5e80b8650 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5e8147080 0x7fe5e80b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 --2- 172.21.15.64:0/2575890120 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5e80b55c0 0x7fe5e81f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 >> 172.21.15.64:0/2575890120 conn(0x7fe5e8071300 msgr2=0x7fe5e81435c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:08.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 shutdown_connections 2024-03-07T07:20:08.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.352+0000 7fe5ee634640 1 -- 172.21.15.64:0/2575890120 wait complete. 2024-03-07T07:20:08.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:20:08.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.464+0000 7f680e7b8640 1 Processor -- start 2024-03-07T07:20:08.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.464+0000 7f680e7b8640 1 -- start start 2024-03-07T07:20:08.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680e7b8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f680806dab0 0x7f6808071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680e7b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68080a9430 0x7f68080afb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680e7b8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680815e650 con 0x7f6808070400 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680e7b8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6808070970 con 0x7f680806dab0 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680e7b8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6808070c50 con 0x7f68080a9430 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680cd55640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68080a9430 0x7f68080afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f6807fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f680806dab0 0x7f6808071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680cd55640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68080a9430 0x7f68080afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:37830/0 (socket says 172.21.15.64:37830) 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f68077fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f6808070400 0x7f680806de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:53840/0 (socket says 172.21.15.64:53840) 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f6807fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f680806dab0 0x7f6808071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:59944/0 (socket says 172.21.15.64:59944) 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.465+0000 7f680cd55640 1 -- 172.21.15.64:0/580272175 learned_addr learned my addr 172.21.15.64:0/580272175 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.466+0000 7f6807fff640 1 -- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68080a9430 msgr2=0x7f68080afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.466+0000 7f6807fff640 1 --2- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68080a9430 0x7f68080afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.466+0000 7f6807fff640 1 -- 172.21.15.64:0/580272175 >> v1:172.21.15.120:6789/0 conn(0x7f6808070400 legacy=0x7f680806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.466+0000 7f6807fff640 1 -- 172.21.15.64:0/580272175 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808070f30 con 0x7f680806dab0 2024-03-07T07:20:08.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.466+0000 7f680cd55640 1 --2- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68080a9430 0x7f68080afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.466+0000 7f6807fff640 1 --2- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f680806dab0 0x7f6808071a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0097a0 tx=0x7f67fc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eea46cf51f60e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f6806ffd640 1 -- 172.21.15.64:0/580272175 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc035020 con 0x7f680806dab0 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f6806ffd640 1 -- 172.21.15.64:0/580272175 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc031830 con 0x7f680806dab0 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f6806ffd640 1 -- 172.21.15.64:0/580272175 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc031c00 con 0x7f680806dab0 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 -- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f680806dab0 msgr2=0x7f6808071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 --2- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f680806dab0 0x7f6808071a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0097a0 tx=0x7f67fc02b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 -- 172.21.15.64:0/580272175 shutdown_connections 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 --2- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f68080a9430 0x7f68080afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 --2- 172.21.15.64:0/580272175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f680806dab0 0x7f6808071a10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 -- 172.21.15.64:0/580272175 >> 172.21.15.64:0/580272175 conn(0x7f68080cd160 msgr2=0x7f68080cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:08.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 -- 172.21.15.64:0/580272175 shutdown_connections 2024-03-07T07:20:08.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.467+0000 7f680e7b8640 1 -- 172.21.15.64:0/580272175 wait complete. 2024-03-07T07:20:08.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.468+0000 7f680e7b8640 1 Processor -- start 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f680e7b8640 1 -- start start 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f680e7b8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f680806dab0 0x7f68080c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f680e7b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6808070400 0x7f6808144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f680e7b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68080a9430 0x7f6808155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f680e7b8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080673b0 con 0x7f680806dab0 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f680e7b8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080714f0 con 0x7f68080a9430 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f680e7b8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6808067530 con 0x7f6808070400 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f6807fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f680806dab0 0x7f68080c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f6807fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f680806dab0 0x7f68080c2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35412/0 (socket says 172.21.15.64:35412) 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.469+0000 7f6807fff640 1 -- 172.21.15.64:0/4286554286 learned_addr learned my addr 172.21.15.64:0/4286554286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:08.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f680cd55640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68080a9430 0x7f6808155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f68077fe640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6808070400 0x7f6808144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6807fff640 1 -- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6808070400 msgr2=0x7f6808144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6807fff640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6808070400 0x7f6808144fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6807fff640 1 -- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68080a9430 msgr2=0x7f6808155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6807fff640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68080a9430 0x7f6808155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6807fff640 1 -- 172.21.15.64:0/4286554286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808145520 con 0x7f680806dab0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f680cd55640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68080a9430 0x7f6808155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6807fff640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f680806dab0 0x7f68080c2e60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0098c0 tx=0x7f67fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6804ff9640 1 -- 172.21.15.64:0/4286554286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc040070 con 0x7f680806dab0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6804ff9640 1 -- 172.21.15.64:0/4286554286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc03e070 con 0x7f680806dab0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6808159f50 con 0x7f680806dab0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f6804ff9640 1 -- 172.21.15.64:0/4286554286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc035040 con 0x7f680806dab0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.470+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68081458e0 con 0x7f680806dab0 2024-03-07T07:20:08.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.471+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6808062c90 con 0x7f680806dab0 2024-03-07T07:20:08.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.472+0000 7f6804ff9640 1 -- 172.21.15.64:0/4286554286 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f67fc038020 con 0x7f680806dab0 2024-03-07T07:20:08.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.472+0000 7f6804ff9640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f67f406b4d0 0x7f67f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.472+0000 7f6804ff9640 1 -- 172.21.15.64:0/4286554286 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67fc0ab700 con 0x7f680806dab0 2024-03-07T07:20:08.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.472+0000 7f68077fe640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f67f406b4d0 0x7f67f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.473+0000 7f68077fe640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f67f406b4d0 0x7f67f406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f680815f270 tx=0x7f67f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.476+0000 7f6804ff9640 1 -- 172.21.15.64:0/4286554286 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f67fc031800 con 0x7f680806dab0 2024-03-07T07:20:08.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.658+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f680806eef0 con 0x7f680806dab0 2024-03-07T07:20:08.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.659+0000 7f6804ff9640 1 -- 172.21.15.64:0/4286554286 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f67fc07a580 con 0x7f680806dab0 2024-03-07T07:20:08.659 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:08.660 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:20:07.375395+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:20:05.640176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T07:20:06.344694+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.661+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f67f406b4d0 msgr2=0x7f67f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.661+0000 7f680e7b8640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f67f406b4d0 0x7f67f406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f680815f270 tx=0x7f67f8009040 comp rx=0 tx=0).stop 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f680806dab0 msgr2=0x7f68080c2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f680806dab0 0x7f68080c2e60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0098c0 tx=0x7f67fc0046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 shutdown_connections 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f67f406b4d0 0x7f67f406d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68080a9430 0x7f6808155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6808070400 0x7f6808144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 --2- 172.21.15.64:0/4286554286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f680806dab0 0x7f68080c2e60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 >> 172.21.15.64:0/4286554286 conn(0x7f68080cd160 msgr2=0x7f68080b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 shutdown_connections 2024-03-07T07:20:08.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.662+0000 7f680e7b8640 1 -- 172.21.15.64:0/4286554286 wait complete. 2024-03-07T07:20:08.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-07T07:20:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.777+0000 7efe1db5a640 1 Processor -- start 2024-03-07T07:20:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.777+0000 7efe1db5a640 1 -- start start 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.777+0000 7efe1db5a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe180cedc0 0x7efe180cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.777+0000 7efe1db5a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe180d2b10 0x7efe180d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.777+0000 7efe1db5a640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180b7190 con 0x7efe1815e270 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.777+0000 7efe1db5a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe180721a0 con 0x7efe180cedc0 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.777+0000 7efe1db5a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe18071be0 con 0x7efe180d2b10 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe16ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7efe180cedc0 0x7efe180cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe177fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7efe1815e270 0x7efe18071110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:56482/0 (socket says 172.21.15.64:56482) 2024-03-07T07:20:08.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe177fe640 1 -- 172.21.15.64:0/3291649869 learned_addr learned my addr 172.21.15.64:0/3291649869 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe16ffd640 1 -- 172.21.15.64:0/3291649869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe180d2b10 msgr2=0x7efe180d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe17fff640 1 --2- 172.21.15.64:0/3291649869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe180d2b10 0x7efe180d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe16ffd640 1 --2- 172.21.15.64:0/3291649869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe180d2b10 0x7efe180d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe16ffd640 1 -- 172.21.15.64:0/3291649869 >> v1:172.21.15.64:6789/0 conn(0x7efe1815e270 legacy=0x7efe18071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe167fc640 1 -- 172.21.15.64:0/3291649869 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359349256 0 0) 0x7efe08001110 con 0x7efe1815e270 2024-03-07T07:20:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe16ffd640 1 -- 172.21.15.64:0/3291649869 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18071ec0 con 0x7efe180cedc0 2024-03-07T07:20:08.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.778+0000 7efe17fff640 1 --2- 172.21.15.64:0/3291649869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe180d2b10 0x7efe180d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:08.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe16ffd640 1 --2- 172.21.15.64:0/3291649869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180cedc0 0x7efe180cf270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0097a0 tx=0x7efe0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2f7334a1abdd59 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe167fc640 1 -- 172.21.15.64:0/3291649869 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0315a0 con 0x7efe180cedc0 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe167fc640 1 -- 172.21.15.64:0/3291649869 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c037070 con 0x7efe180cedc0 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe167fc640 1 -- 172.21.15.64:0/3291649869 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c031960 con 0x7efe180cedc0 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe1db5a640 1 -- 172.21.15.64:0/3291649869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180cedc0 msgr2=0x7efe180cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe1db5a640 1 --2- 172.21.15.64:0/3291649869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180cedc0 0x7efe180cf270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0097a0 tx=0x7efe0c02b840 comp rx=0 tx=0).stop 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe1db5a640 1 -- 172.21.15.64:0/3291649869 shutdown_connections 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe1db5a640 1 --2- 172.21.15.64:0/3291649869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe180d2b10 0x7efe180d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe1db5a640 1 --2- 172.21.15.64:0/3291649869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180cedc0 0x7efe180cf270 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe1db5a640 1 -- 172.21.15.64:0/3291649869 >> 172.21.15.64:0/3291649869 conn(0x7efe180d0e90 msgr2=0x7efe180d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.779+0000 7efe1db5a640 1 -- 172.21.15.64:0/3291649869 shutdown_connections 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.780+0000 7efe1db5a640 1 -- 172.21.15.64:0/3291649869 wait complete. 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 Processor -- start 2024-03-07T07:20:08.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 -- start start 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe180cedc0 0x7efe180c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180d2b10 0x7efe180cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe1815e270 0x7efe181e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1815e650 con 0x7efe180d2b10 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180d25f0 con 0x7efe180cedc0 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.781+0000 7efe1db5a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe18071820 con 0x7efe1815e270 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180d2b10 0x7efe180cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180d2b10 0x7efe180cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35430/0 (socket says 172.21.15.64:35430) 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 -- 172.21.15.64:0/1086386322 learned_addr learned my addr 172.21.15.64:0/1086386322 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe177fe640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe180cedc0 0x7efe180c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe17fff640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe1815e270 0x7efe181e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 -- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe1815e270 msgr2=0x7efe181e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe1815e270 0x7efe181e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 -- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe180cedc0 msgr2=0x7efe180c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe180cedc0 0x7efe180c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 -- 172.21.15.64:0/1086386322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180c9440 con 0x7efe180d2b10 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe177fe640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe180cedc0 0x7efe180c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe16ffd640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180d2b10 0x7efe180cc6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe0c034040 tx=0x7efe0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efdfffff640 1 -- 172.21.15.64:0/1086386322 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c032840 con 0x7efe180d2b10 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efdfffff640 1 -- 172.21.15.64:0/1086386322 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c0329e0 con 0x7efe180d2b10 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efdfffff640 1 -- 172.21.15.64:0/1086386322 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c032dd0 con 0x7efe180d2b10 2024-03-07T07:20:08.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe180ccf90 con 0x7efe180d2b10 2024-03-07T07:20:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.782+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe180cd550 con 0x7efe180d2b10 2024-03-07T07:20:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.783+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18062c90 con 0x7efe180d2b10 2024-03-07T07:20:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.784+0000 7efdfffff640 1 -- 172.21.15.64:0/1086386322 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efe0c04e430 con 0x7efe180d2b10 2024-03-07T07:20:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.784+0000 7efdfffff640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7efe0406b4d0 0x7efe0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.784+0000 7efdfffff640 1 -- 172.21.15.64:0/1086386322 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe0c02f030 con 0x7efe180d2b10 2024-03-07T07:20:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.784+0000 7efe177fe640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7efe0406b4d0 0x7efe0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:08.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.785+0000 7efe177fe640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7efe0406b4d0 0x7efe0406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe08000f60 tx=0x7efe08007cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:08.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:08.788+0000 7efdfffff640 1 -- 172.21.15.64:0/1086386322 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efe0c040070 con 0x7efe180d2b10 2024-03-07T07:20:09.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.009+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efe18070820 con 0x7efe180d2b10 2024-03-07T07:20:09.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.034+0000 7efdfffff640 1 -- 172.21.15.64:0/1086386322 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7efe0c07b110 con 0x7efe180d2b10 2024-03-07T07:20:09.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7efe0406b4d0 msgr2=0x7efe0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:09.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7efe0406b4d0 0x7efe0406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe08000f60 tx=0x7efe08007cc0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180d2b10 msgr2=0x7efe180cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180d2b10 0x7efe180cc6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe0c034040 tx=0x7efe0c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 shutdown_connections 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7efe0406b4d0 0x7efe0406d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe180d2b10 0x7efe180cc6b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe1815e270 0x7efe181e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 --2- 172.21.15.64:0/1086386322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe180cedc0 0x7efe180c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 >> 172.21.15.64:0/1086386322 conn(0x7efe180d0e90 msgr2=0x7efe180b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 shutdown_connections 2024-03-07T07:20:09.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.036+0000 7efe1db5a640 1 -- 172.21.15.64:0/1086386322 wait complete. 2024-03-07T07:20:09.050 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-07T07:20:09.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-07T07:20:09.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.157+0000 7f88ae5a7640 1 Processor -- start 2024-03-07T07:20:09.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.157+0000 7f88ae5a7640 1 -- start start 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88ae5a7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f88a815e270 0x7f88a8071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88ae5a7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a8071650 0x7f88a80cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88ae5a7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88a80cf300 0x7f88a8161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88ae5a7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80cf750 con 0x7f88a80cf300 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88ae5a7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80721a0 con 0x7f88a815e270 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88ae5a7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80b7190 con 0x7f88a8071650 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88acb44640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88a80cf300 0x7f88a8161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88acb44640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88a80cf300 0x7f88a8161dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35438/0 (socket says 172.21.15.64:35438) 2024-03-07T07:20:09.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.158+0000 7f88acb44640 1 -- 172.21.15.64:0/1262547753 learned_addr learned my addr 172.21.15.64:0/1262547753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88a7fff640 1 --2- 172.21.15.64:0/1262547753 >> v2:172.21.15.120:3300/0 conn(0x7f88a815e270 0x7f88a8071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88a77fe640 1 --2- 172.21.15.64:0/1262547753 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a8071650 0x7f88a80cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88acb44640 1 -- 172.21.15.64:0/1262547753 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a8071650 msgr2=0x7f88a80cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88acb44640 1 --2- 172.21.15.64:0/1262547753 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a8071650 0x7f88a80cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88acb44640 1 -- 172.21.15.64:0/1262547753 >> v2:172.21.15.120:3300/0 conn(0x7f88a815e270 msgr2=0x7f88a8071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88acb44640 1 --2- 172.21.15.64:0/1262547753 >> v2:172.21.15.120:3300/0 conn(0x7f88a815e270 0x7f88a8071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88acb44640 1 -- 172.21.15.64:0/1262547753 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a8071be0 con 0x7f88a80cf300 2024-03-07T07:20:09.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.159+0000 7f88a7fff640 1 --2- 172.21.15.64:0/1262547753 >> v2:172.21.15.120:3300/0 conn(0x7f88a815e270 0x7f88a8071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.160+0000 7f88acb44640 1 --2- 172.21.15.64:0/1262547753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80cf300 0x7f88a8161dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f88a808d200 tx=0x7f88a002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d43e2adb29ff690 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.160+0000 7f88a6ffd640 1 -- 172.21.15.64:0/1262547753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a002c9e0 con 0x7f88a80cf300 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.160+0000 7f88a6ffd640 1 -- 172.21.15.64:0/1262547753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a002cb80 con 0x7f88a80cf300 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.160+0000 7f88a6ffd640 1 -- 172.21.15.64:0/1262547753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00343f0 con 0x7f88a80cf300 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1262547753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80cf300 msgr2=0x7f88a8161dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1262547753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80cf300 0x7f88a8161dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f88a808d200 tx=0x7f88a002be40 comp rx=0 tx=0).stop 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1262547753 shutdown_connections 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1262547753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80cf300 0x7f88a8161dd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1262547753 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a8071650 0x7f88a80cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1262547753 >> v2:172.21.15.120:3300/0 conn(0x7f88a815e270 0x7f88a8071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1262547753 >> 172.21.15.64:0/1262547753 conn(0x7f88a80d0e90 msgr2=0x7f88a80d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:09.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1262547753 shutdown_connections 2024-03-07T07:20:09.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.161+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1262547753 wait complete. 2024-03-07T07:20:09.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.162+0000 7f88ae5a7640 1 Processor -- start 2024-03-07T07:20:09.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.162+0000 7f88ae5a7640 1 -- start start 2024-03-07T07:20:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88ae5a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8071650 0x7f88a8160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88ae5a7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a80cf300 0x7f88a80ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88ae5a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a815e270 0x7f88a80ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88ae5a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80d25f0 con 0x7f88a815e270 2024-03-07T07:20:09.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88ae5a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80d2310 con 0x7f88a8071650 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88ae5a7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a815e650 con 0x7f88a80cf300 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a815e270 0x7f88a80ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a815e270 0x7f88a80ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35446/0 (socket says 172.21.15.64:35446) 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 -- 172.21.15.64:0/1442779692 learned_addr learned my addr 172.21.15.64:0/1442779692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88a77fe640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a80cf300 0x7f88a80ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88a7fff640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8071650 0x7f88a8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 -- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a80cf300 msgr2=0x7f88a80ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a80cf300 0x7f88a80ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 -- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8071650 msgr2=0x7f88a8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8071650 0x7f88a8160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 -- 172.21.15.64:0/1442779692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80cbf10 con 0x7f88a815e270 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88acb44640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a815e270 0x7f88a80ca630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f88a0036040 tx=0x7f88a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:09.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88a4ff9640 1 -- 172.21.15.64:0/1442779692 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0043020 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.163+0000 7f88a4ff9640 1 -- 172.21.15.64:0/1442779692 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a003e450 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.164+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a80cc1f0 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.164+0000 7f88a4ff9640 1 -- 172.21.15.64:0/1442779692 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0009510 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.164+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a80cc7b0 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.164+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a80cc9f0 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.165+0000 7f88a4ff9640 1 -- 172.21.15.64:0/1442779692 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f88a0046020 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.165+0000 7f88a4ff9640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.165+0000 7f88a4ff9640 1 -- 172.21.15.64:0/1442779692 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88a00ac900 con 0x7f88a815e270 2024-03-07T07:20:09.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.166+0000 7f88a7fff640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:09.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.166+0000 7f88a7fff640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8898006fd0 tx=0x7f8898008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:09.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.170+0000 7f88a4ff9640 1 -- 172.21.15.64:0/1442779692 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f88a80cc9f0 con 0x7f88a815e270 2024-03-07T07:20:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:09.483+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f88a8062c90 con 0x7f88a815e270 2024-03-07T07:20:10.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.035+0000 7f88a4ff9640 1 -- 172.21.15.64:0/1442779692 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88a004d030 con 0x7f88a815e270 2024-03-07T07:20:10.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 msgr2=0x7f889406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8898006fd0 tx=0x7f8898008040 comp rx=0 tx=0).stop 2024-03-07T07:20:10.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a815e270 msgr2=0x7f88a80ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a815e270 0x7f88a80ca630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f88a0036040 tx=0x7f88a0004a40 comp rx=0 tx=0).stop 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 shutdown_connections 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a815e270 0x7f88a80ca630 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a80cf300 0x7f88a80ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 --2- 172.21.15.64:0/1442779692 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8071650 0x7f88a8160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 >> 172.21.15.64:0/1442779692 conn(0x7f88a80d0e90 msgr2=0x7f88a80d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 shutdown_connections 2024-03-07T07:20:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.038+0000 7f88ae5a7640 1 -- 172.21.15.64:0/1442779692 wait complete. 2024-03-07T07:20:10.049 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-07T07:20:10.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:20:10.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.151+0000 7f2cd6f61640 1 Processor -- start 2024-03-07T07:20:10.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.151+0000 7f2cd6f61640 1 -- start start 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd6f61640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2cd0147270 0x7f2cd006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd6f61640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cd0147810 0x7f2cd006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd6f61640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd00d1e70 con 0x7f2cd00d2c10 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd6f61640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd0147650 con 0x7f2cd0147810 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd6f61640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd00b7520 con 0x7f2cd0147270 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd54fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cd0147810 0x7f2cd006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd54fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f2cd0147810 0x7f2cd006ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35450/0 (socket says 172.21.15.64:35450) 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.152+0000 7f2cd54fe640 1 -- 172.21.15.64:0/567738458 learned_addr learned my addr 172.21.15.64:0/567738458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.153+0000 7f2cd4cfd640 1 --2- 172.21.15.64:0/567738458 >> v2:172.21.15.120:3300/0 conn(0x7f2cd0147270 0x7f2cd006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.153+0000 7f2cd54fe640 1 -- 172.21.15.64:0/567738458 >> v2:172.21.15.120:3300/0 conn(0x7f2cd0147270 msgr2=0x7f2cd006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.153+0000 7f2cd54fe640 1 --2- 172.21.15.64:0/567738458 >> v2:172.21.15.120:3300/0 conn(0x7f2cd0147270 0x7f2cd006b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.153+0000 7f2cd54fe640 1 -- 172.21.15.64:0/567738458 >> v1:172.21.15.120:6789/0 conn(0x7f2cd00d2c10 legacy=0x7f2cd0162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:10.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.153+0000 7f2cd54fe640 1 -- 172.21.15.64:0/567738458 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd00d2130 con 0x7f2cd0147810 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.153+0000 7f2cd54fe640 1 --2- 172.21.15.64:0/567738458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd0147810 0x7f2cd006ba50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2cd006dd60 tx=0x7f2cc802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3885164a72602c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2ccf7fe640 1 -- 172.21.15.64:0/567738458 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc802ca80 con 0x7f2cd0147810 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2ccf7fe640 1 -- 172.21.15.64:0/567738458 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc802cc20 con 0x7f2cd0147810 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2ccf7fe640 1 -- 172.21.15.64:0/567738458 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80343f0 con 0x7f2cd0147810 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 -- 172.21.15.64:0/567738458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd0147810 msgr2=0x7f2cd006ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/567738458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd0147810 0x7f2cd006ba50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2cd006dd60 tx=0x7f2cc802bee0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 -- 172.21.15.64:0/567738458 shutdown_connections 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/567738458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd0147810 0x7f2cd006ba50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/567738458 >> v2:172.21.15.120:3300/0 conn(0x7f2cd0147270 0x7f2cd006b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 -- 172.21.15.64:0/567738458 >> 172.21.15.64:0/567738458 conn(0x7f2cd006d4c0 msgr2=0x7f2cd0066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 -- 172.21.15.64:0/567738458 shutdown_connections 2024-03-07T07:20:10.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.154+0000 7f2cd6f61640 1 -- 172.21.15.64:0/567738458 wait complete. 2024-03-07T07:20:10.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.155+0000 7f2cd6f61640 1 Processor -- start 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.155+0000 7f2cd6f61640 1 -- start start 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd6f61640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd00d2c10 0x7f2cd01f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd6f61640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd0147270 0x7f2cd01f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd6f61640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2cd0147810 0x7f2cd015f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd6f61640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00b71d0 con 0x7f2cd00d2c10 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd6f61640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd00d26f0 con 0x7f2cd0147270 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd6f61640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0147d60 con 0x7f2cd0147810 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd4cfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd00d2c10 0x7f2cd01f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd4cfd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd00d2c10 0x7f2cd01f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35466/0 (socket says 172.21.15.64:35466) 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.156+0000 7f2cd4cfd640 1 -- 172.21.15.64:0/1554769614 learned_addr learned my addr 172.21.15.64:0/1554769614 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2ccffff640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd0147270 0x7f2cd01f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd54fe640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2cd0147810 0x7f2cd015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd4cfd640 1 -- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2cd0147810 msgr2=0x7f2cd015f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd4cfd640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2cd0147810 0x7f2cd015f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd4cfd640 1 -- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd0147270 msgr2=0x7f2cd01f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd4cfd640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd0147270 0x7f2cd01f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd4cfd640 1 -- 172.21.15.64:0/1554769614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd015fe80 con 0x7f2cd00d2c10 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd4cfd640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd00d2c10 0x7f2cd01f40e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d800 tx=0x7f2cc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2ccd7fa640 1 -- 172.21.15.64:0/1554769614 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400b660 con 0x7f2cd00d2c10 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2ccd7fa640 1 -- 172.21.15.64:0/1554769614 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc4019070 con 0x7f2cd00d2c10 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd0160160 con 0x7f2cd00d2c10 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2ccd7fa640 1 -- 172.21.15.64:0/1554769614 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400ba20 con 0x7f2cd00d2c10 2024-03-07T07:20:10.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.157+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd0160720 con 0x7f2cd00d2c10 2024-03-07T07:20:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.158+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd0062c90 con 0x7f2cd00d2c10 2024-03-07T07:20:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.159+0000 7f2ccd7fa640 1 -- 172.21.15.64:0/1554769614 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2cc400bbc0 con 0x7f2cd00d2c10 2024-03-07T07:20:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.159+0000 7f2ccd7fa640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2cb406b4d0 0x7f2cb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.159+0000 7f2ccd7fa640 1 -- 172.21.15.64:0/1554769614 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cc408a490 con 0x7f2cd00d2c10 2024-03-07T07:20:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.160+0000 7f2ccffff640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2cb406b4d0 0x7f2cb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.160+0000 7f2ccffff640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2cb406b4d0 0x7f2cb406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0146630 tx=0x7f2cbc007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.163+0000 7f2ccd7fa640 1 -- 172.21.15.64:0/1554769614 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2cc4058d50 con 0x7f2cd00d2c10 2024-03-07T07:20:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.380+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cd0071cc0 con 0x7f2cd00d2c10 2024-03-07T07:20:10.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.381+0000 7f2ccd7fa640 1 -- 172.21.15.64:0/1554769614 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f2cc405cdf0 con 0x7f2cd00d2c10 2024-03-07T07:20:10.381 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:10.381 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/1229777071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1229777071},{"type":"v1","addr":"172.21.15.120:6836","nonce":1229777071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3785220969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":3785220969},{"type":"v1","addr":"172.21.15.120:6837","nonce":3785220969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/229489259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":229489259},{"type":"v1","addr":"172.21.15.64:6834","nonce":229489259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:20:07.375701+0000","modified":"2024-03-07T07:20:09.484751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3131353555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3131353555},{"type":"v1","addr":"172.21.15.64:6835","nonce":3131353555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-03-07T07:20:10.381 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2cb406b4d0 msgr2=0x7f2cb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2cb406b4d0 0x7f2cb406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0146630 tx=0x7f2cbc007e30 comp rx=0 tx=0).stop 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd00d2c10 msgr2=0x7f2cd01f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd00d2c10 0x7f2cd01f40e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d800 tx=0x7f2cc400dcc0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 shutdown_connections 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f2cb406b4d0 0x7f2cb406d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2cd0147810 0x7f2cd015f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd0147270 0x7f2cd01f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 --2- 172.21.15.64:0/1554769614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2cd00d2c10 0x7f2cd01f40e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 >> 172.21.15.64:0/1554769614 conn(0x7f2cd006d4c0 msgr2=0x7f2cd00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 shutdown_connections 2024-03-07T07:20:10.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.383+0000 7f2cd6f61640 1 -- 172.21.15.64:0/1554769614 wait complete. 2024-03-07T07:20:10.397 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:20:10.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.499+0000 7f4d9b26b640 1 Processor -- start 2024-03-07T07:20:10.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.499+0000 7f4d9b26b640 1 -- start start 2024-03-07T07:20:10.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d9b26b640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d940d1860 0x7f4d9406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d9b26b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4d940b7390 0x7f4d9406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d9b26b640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d940d0aa0 con 0x7f4d94145bf0 2024-03-07T07:20:10.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d9b26b640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d940d13d0 con 0x7f4d940d1860 2024-03-07T07:20:10.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d9b26b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d940d0d80 con 0x7f4d940b7390 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d98806640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f4d940d1860 0x7f4d9406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d99007640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4d94145bf0 0x7f4d940cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60138/0 (socket says 172.21.15.64:60138) 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d99007640 1 -- 172.21.15.64:0/1898976925 learned_addr learned my addr 172.21.15.64:0/1898976925 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d99808640 1 --2- 172.21.15.64:0/1898976925 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b7390 0x7f4d9406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d98806640 1 -- 172.21.15.64:0/1898976925 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b7390 msgr2=0x7f4d9406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d98806640 1 --2- 172.21.15.64:0/1898976925 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b7390 0x7f4d9406ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d98806640 1 -- 172.21.15.64:0/1898976925 >> v1:172.21.15.64:6789/0 conn(0x7f4d94145bf0 legacy=0x7f4d940cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d8bfff640 1 -- 172.21.15.64:0/1898976925 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361840471 0 0) 0x7f4d940d0aa0 con 0x7f4d94145bf0 2024-03-07T07:20:10.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.500+0000 7f4d98806640 1 -- 172.21.15.64:0/1898976925 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d940d1060 con 0x7f4d940d1860 2024-03-07T07:20:10.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.501+0000 7f4d98806640 1 --2- 172.21.15.64:0/1898976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d940d1860 0x7f4d9406b510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d940b4620 tx=0x7f4d8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3abfc4ed1df0aa server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.501+0000 7f4d8bfff640 1 -- 172.21.15.64:0/1898976925 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84035020 con 0x7f4d940d1860 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.501+0000 7f4d8bfff640 1 -- 172.21.15.64:0/1898976925 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d84031830 con 0x7f4d940d1860 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.501+0000 7f4d8bfff640 1 -- 172.21.15.64:0/1898976925 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d84031c00 con 0x7f4d940d1860 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.501+0000 7f4d9b26b640 1 -- 172.21.15.64:0/1898976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d940d1860 msgr2=0x7f4d9406b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.501+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/1898976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d940d1860 0x7f4d9406b510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d940b4620 tx=0x7f4d8402b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.502+0000 7f4d9b26b640 1 -- 172.21.15.64:0/1898976925 shutdown_connections 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.502+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/1898976925 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b7390 0x7f4d9406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.502+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/1898976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d940d1860 0x7f4d9406b510 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.502+0000 7f4d9b26b640 1 -- 172.21.15.64:0/1898976925 >> 172.21.15.64:0/1898976925 conn(0x7f4d9406d4c0 msgr2=0x7f4d94066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.502+0000 7f4d9b26b640 1 -- 172.21.15.64:0/1898976925 shutdown_connections 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.502+0000 7f4d9b26b640 1 -- 172.21.15.64:0/1898976925 wait complete. 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 Processor -- start 2024-03-07T07:20:10.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 -- start start 2024-03-07T07:20:10.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d940b7390 0x7f4d9415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940d1860 0x7f4d94072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d94145bf0 0x7f4d94071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940d0840 con 0x7f4d94145bf0 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940d06c0 con 0x7f4d940d1860 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.503+0000 7f4d9b26b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940b5160 con 0x7f4d940b7390 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d94145bf0 0x7f4d94071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d94145bf0 0x7f4d94071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60272/0 (socket says 172.21.15.64:60272) 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 -- 172.21.15.64:0/2123308312 learned_addr learned my addr 172.21.15.64:0/2123308312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d98806640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940d1860 0x7f4d94072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99007640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d940b7390 0x7f4d9415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 -- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d940b7390 msgr2=0x7f4d9415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d940b7390 0x7f4d9415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 -- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940d1860 msgr2=0x7f4d94072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940d1860 0x7f4d94072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 -- 172.21.15.64:0/2123308312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9415ed20 con 0x7f4d94145bf0 2024-03-07T07:20:10.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d98806640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940d1860 0x7f4d94072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99007640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d940b7390 0x7f4d9415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d99808640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d94145bf0 0x7f4d94071550 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c007e40 tx=0x7f4d8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d89ffb640 1 -- 172.21.15.64:0/2123308312 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c014620 con 0x7f4d94145bf0 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d89ffb640 1 -- 172.21.15.64:0/2123308312 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c019070 con 0x7f4d94145bf0 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d89ffb640 1 -- 172.21.15.64:0/2123308312 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c014a20 con 0x7f4d94145bf0 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d94072bc0 con 0x7f4d94145bf0 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.504+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941f4580 con 0x7f4d94145bf0 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.506+0000 7f4d89ffb640 1 -- 172.21.15.64:0/2123308312 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4d8c014bc0 con 0x7f4d94145bf0 2024-03-07T07:20:10.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.506+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d94062c90 con 0x7f4d94145bf0 2024-03-07T07:20:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.506+0000 7f4d89ffb640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4d7806b4d0 0x7f4d7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.506+0000 7f4d89ffb640 1 -- 172.21.15.64:0/2123308312 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d8c08ab40 con 0x7f4d94145bf0 2024-03-07T07:20:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.507+0000 7f4d99007640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4d7806b4d0 0x7f4d7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.507+0000 7f4d99007640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4d7806b4d0 0x7f4d7806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000f60 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.510+0000 7f4d89ffb640 1 -- 172.21.15.64:0/2123308312 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d8c0599c0 con 0x7f4d94145bf0 2024-03-07T07:20:10.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.713+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d94072df0 con 0x7f4d94145bf0 2024-03-07T07:20:10.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.714+0000 7f4d89ffb640 1 -- 172.21.15.64:0/2123308312 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f4d8c0592c0 con 0x7f4d94145bf0 2024-03-07T07:20:10.714 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:10.714 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:20:09.012362+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:20:05.640176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T07:20:06.344694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4d7806b4d0 msgr2=0x7f4d7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4d7806b4d0 0x7f4d7806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000f60 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d94145bf0 msgr2=0x7f4d94071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d94145bf0 0x7f4d94071550 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c007e40 tx=0x7f4d8c00e4b0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 shutdown_connections 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4d7806b4d0 0x7f4d7806d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d94145bf0 0x7f4d94071550 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940d1860 0x7f4d94072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 --2- 172.21.15.64:0/2123308312 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d940b7390 0x7f4d9415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 >> 172.21.15.64:0/2123308312 conn(0x7f4d9406d4c0 msgr2=0x7f4d9406b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:10.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 shutdown_connections 2024-03-07T07:20:10.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.716+0000 7f4d9b26b640 1 -- 172.21.15.64:0/2123308312 wait complete. 2024-03-07T07:20:10.727 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-07T07:20:10.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:20:10.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.828+0000 7fea05894640 1 Processor -- start 2024-03-07T07:20:10.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.828+0000 7fea05894640 1 -- start start 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.828+0000 7fea05894640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fea000b6120 0x7fea00070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.828+0000 7fea05894640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 0x7fea000ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.828+0000 7fea05894640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea00147ea0 con 0x7fea000b43f0 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.828+0000 7fea05894640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea00071390 con 0x7fea000b6120 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.828+0000 7fea05894640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea000b4960 con 0x7fea000ae470 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fe7fc640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fea000b43f0 0x7fea000b6500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60148/0 (socket says 172.21.15.64:60148) 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fe7fc640 1 -- 172.21.15.64:0/750257932 learned_addr learned my addr 172.21.15.64:0/750257932 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:10.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9feffd640 1 --2- 172.21.15.64:0/750257932 >> v2:172.21.15.120:3300/0 conn(0x7fea000b6120 0x7fea00070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 0x7fea000ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358410049 0 0) 0x7fe9f4001110 con 0x7fea000b43f0 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e8003520 con 0x7fea000b43f0 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 403388085 0 0) 0x7fe9e8003520 con 0x7fea000b43f0 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 msgr2=0x7fea000ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 0x7fea000ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 >> v2:172.21.15.120:3300/0 conn(0x7fea000b6120 msgr2=0x7fea00070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 --2- 172.21.15.64:0/750257932 >> v2:172.21.15.120:3300/0 conn(0x7fea000b6120 0x7fea00070e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000b4c40 con 0x7fea000b43f0 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7fe9f4001840 con 0x7fea000b43f0 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 >> v1:172.21.15.64:6789/0 conn(0x7fea000b43f0 legacy=0x7fea000b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9e80039b0 0x7fe9e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 0x7fe9e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00071390 con 0x7fe9e8004d40 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea000b4960 con 0x7fe9e80039b0 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e800b9c0 con 0x7fea000ae470 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9feffd640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9e80039b0 0x7fe9e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.829+0000 7fe9fe7fc640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 0x7fe9e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.830+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 msgr2=0x7fe9e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.830+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea000ae470 0x7fe9e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.830+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9e80039b0 msgr2=0x7fe9e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.830+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe9e80039b0 0x7fe9e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.830+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/750257932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8003520 con 0x7fe9e8004d40 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.830+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e170 tx=0x7fe9e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74892e0de23a39d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8039070 con 0x7fe9e8004d40 2024-03-07T07:20:10.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f802cbf0 con 0x7fe9e8004d40 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fe9fdffb640 1 -- 172.21.15.64:0/750257932 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8009c10 con 0x7fe9e8004d40 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fea05894640 1 -- 172.21.15.64:0/750257932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8004d40 msgr2=0x7fe9e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fea05894640 1 --2- 172.21.15.64:0/750257932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e170 tx=0x7fe9e8005830 comp rx=0 tx=0).stop 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/750257932 reap_dead start 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fea05894640 1 -- 172.21.15.64:0/750257932 shutdown_connections 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fea05894640 1 -- 172.21.15.64:0/750257932 >> 172.21.15.64:0/750257932 conn(0x7fea00071550 msgr2=0x7fea00071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fea05894640 1 -- 172.21.15.64:0/750257932 shutdown_connections 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.831+0000 7fea05894640 1 -- 172.21.15.64:0/750257932 wait complete. 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.833+0000 7fea05894640 1 Processor -- start 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.833+0000 7fea05894640 1 -- start start 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fea05894640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea001438d0 0x7fea001f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fea05894640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea001e4f80 0x7fea000ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fea05894640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea000cefe0 0x7fea000b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fea05894640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea000ae900 con 0x7fea000cefe0 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fea05894640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea000b5200 con 0x7fea001438d0 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fea05894640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea000aea80 con 0x7fea001e4f80 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fe9ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea000cefe0 0x7fea000b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fe9ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea000cefe0 0x7fea000b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60296/0 (socket says 172.21.15.64:60296) 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.834+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/1990778956 learned_addr learned my addr 172.21.15.64:0/1990778956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9feffd640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea001438d0 0x7fea001f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9fe7fc640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea001e4f80 0x7fea000ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea001e4f80 msgr2=0x7fea000ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea001e4f80 0x7fea000ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea001438d0 msgr2=0x7fea001f7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea001438d0 0x7fea001f7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9ff7fe640 1 -- 172.21.15.64:0/1990778956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea001f8360 con 0x7fea000cefe0 2024-03-07T07:20:10.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9feffd640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea001438d0 0x7fea001f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9ff7fe640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea000cefe0 0x7fea000b05f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009f90 tx=0x7fe9f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1990778956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8045070 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1990778956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f803a040 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea001e3ee0 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1990778956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8039070 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.835+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea001e44a0 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.836+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea00145310 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.837+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1990778956 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe9f8005d10 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.837+0000 7fe9e7fff640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.837+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1990778956 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9f80ab6f0 con 0x7fea000cefe0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.837+0000 7fe9feffd640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.838+0000 7fe9feffd640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009bc0 tx=0x7fe9f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:10.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:10.844+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1990778956 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea00145310 con 0x7fea000cefe0 2024-03-07T07:20:11.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.023+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea00062c90 con 0x7fe9ec06b4d0 2024-03-07T07:20:11.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.030+0000 7fe9e7fff640 1 -- 172.21.15.64:0/1990778956 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251900 (secure 0 0 0) 0x7fea00062c90 con 0x7fe9ec06b4d0 2024-03-07T07:20:11.033 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:11.034 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:20:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe9ec06b4d0 msgr2=0x7fe9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009bc0 tx=0x7fe9f0008040 comp rx=0 tx=0).stop 2024-03-07T07:20:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea000cefe0 msgr2=0x7fea000b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea000cefe0 0x7fea000b05f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009f90 tx=0x7fe9f8009750 comp rx=0 tx=0).stop 2024-03-07T07:20:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 shutdown_connections 2024-03-07T07:20:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea000cefe0 0x7fea000b05f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea001e4f80 0x7fea000ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 --2- 172.21.15.64:0/1990778956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea001438d0 0x7fea001f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 >> 172.21.15.64:0/1990778956 conn(0x7fea00071550 msgr2=0x7fea000b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 shutdown_connections 2024-03-07T07:20:11.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.033+0000 7fea05894640 1 -- 172.21.15.64:0/1990778956 wait complete. 2024-03-07T07:20:11.045 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-03-07T07:20:10.189949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220208,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754512,"statfs":{"total":773094113280,"available":772868620288,"internally_reserved":0,"allocated":4136960,"data_stored":2308084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.182899"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018002+0000","last_change":"2024-03-07T07:20:08.025425+0000","last_active":"2024-03-07T07:20:09.018002+0000","last_peered":"2024-03-07T07:20:09.018002+0000","last_clean":"2024-03-07T07:20:09.018002+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:09.018002+0000","last_undegraded":"2024-03-07T07:20:09.018002+0000","last_fullsized":"2024-03-07T07:20:09.018002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042407+0000","last_change":"2024-03-07T07:20:08.036714+0000","last_active":"2024-03-07T07:20:08.042407+0000","last_peered":"2024-03-07T07:20:08.042407+0000","last_clean":"2024-03-07T07:20:08.042407+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:08.042407+0000","last_undegraded":"2024-03-07T07:20:08.042407+0000","last_fullsized":"2024-03-07T07:20:08.042407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041235+0000","last_change":"2024-03-07T07:20:08.037973+0000","last_active":"2024-03-07T07:20:08.041235+0000","last_peered":"2024-03-07T07:20:08.041235+0000","last_clean":"2024-03-07T07:20:08.041235+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:08.041235+0000","last_undegraded":"2024-03-07T07:20:08.041235+0000","last_fullsized":"2024-03-07T07:20:08.041235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.045882+0000","last_change":"2024-03-07T07:20:08.042993+0000","last_active":"2024-03-07T07:20:08.045882+0000","last_peered":"2024-03-07T07:20:08.045882+0000","last_clean":"2024-03-07T07:20:08.045882+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:08.045882+0000","last_undegraded":"2024-03-07T07:20:08.045882+0000","last_fullsized":"2024-03-07T07:20:08.045882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043121+0000","last_change":"2024-03-07T07:20:08.037872+0000","last_active":"2024-03-07T07:20:08.043121+0000","last_peered":"2024-03-07T07:20:08.043121+0000","last_clean":"2024-03-07T07:20:08.043121+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:08.043121+0000","last_undegraded":"2024-03-07T07:20:08.043121+0000","last_fullsized":"2024-03-07T07:20:08.043121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:53.617691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032465+0000","last_change":"2024-03-07T07:20:08.028212+0000","last_active":"2024-03-07T07:20:08.032465+0000","last_peered":"2024-03-07T07:20:08.032465+0000","last_clean":"2024-03-07T07:20:08.032465+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:08.032465+0000","last_undegraded":"2024-03-07T07:20:08.032465+0000","last_fullsized":"2024-03-07T07:20:08.032465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038277+0000","last_change":"2024-03-07T07:20:08.032379+0000","last_active":"2024-03-07T07:20:08.038277+0000","last_peered":"2024-03-07T07:20:08.038277+0000","last_clean":"2024-03-07T07:20:08.038277+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:08.038277+0000","last_undegraded":"2024-03-07T07:20:08.038277+0000","last_fullsized":"2024-03-07T07:20:08.038277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039373+0000","last_change":"2024-03-07T07:20:08.035495+0000","last_active":"2024-03-07T07:20:08.039373+0000","last_peered":"2024-03-07T07:20:08.039373+0000","last_clean":"2024-03-07T07:20:08.039373+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:08.039373+0000","last_undegraded":"2024-03-07T07:20:08.039373+0000","last_fullsized":"2024-03-07T07:20:08.039373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018731+0000","last_change":"2024-03-07T07:20:08.034012+0000","last_active":"2024-03-07T07:20:09.018731+0000","last_peered":"2024-03-07T07:20:09.018731+0000","last_clean":"2024-03-07T07:20:09.018731+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:09.018731+0000","last_undegraded":"2024-03-07T07:20:09.018731+0000","last_fullsized":"2024-03-07T07:20:09.018731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.044040+0000","last_change":"2024-03-07T07:20:08.037279+0000","last_active":"2024-03-07T07:20:08.044040+0000","last_peered":"2024-03-07T07:20:08.044040+0000","last_clean":"2024-03-07T07:20:08.044040+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:08.044040+0000","last_undegraded":"2024-03-07T07:20:08.044040+0000","last_fullsized":"2024-03-07T07:20:08.044040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041719+0000","last_change":"2024-03-07T07:20:08.039682+0000","last_active":"2024-03-07T07:20:08.041719+0000","last_peered":"2024-03-07T07:20:08.041719+0000","last_clean":"2024-03-07T07:20:08.041719+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:08.041719+0000","last_undegraded":"2024-03-07T07:20:08.041719+0000","last_fullsized":"2024-03-07T07:20:08.041719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038659+0000","last_change":"2024-03-07T07:20:08.028305+0000","last_active":"2024-03-07T07:20:08.038659+0000","last_peered":"2024-03-07T07:20:08.038659+0000","last_clean":"2024-03-07T07:20:08.038659+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:08.038659+0000","last_undegraded":"2024-03-07T07:20:08.038659+0000","last_fullsized":"2024-03-07T07:20:08.038659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034862+0000","last_change":"2024-03-07T07:20:08.033602+0000","last_active":"2024-03-07T07:20:08.034862+0000","last_peered":"2024-03-07T07:20:08.034862+0000","last_clean":"2024-03-07T07:20:08.034862+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:08.034862+0000","last_undegraded":"2024-03-07T07:20:08.034862+0000","last_fullsized":"2024-03-07T07:20:08.034862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:25.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.030215+0000","last_change":"2024-03-07T07:20:08.022351+0000","last_active":"2024-03-07T07:20:08.030215+0000","last_peered":"2024-03-07T07:20:08.030215+0000","last_clean":"2024-03-07T07:20:08.030215+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:08.030215+0000","last_undegraded":"2024-03-07T07:20:08.030215+0000","last_fullsized":"2024-03-07T07:20:08.030215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.024551+0000","last_change":"2024-03-07T07:20:08.021148+0000","last_active":"2024-03-07T07:20:08.024551+0000","last_peered":"2024-03-07T07:20:08.024551+0000","last_clean":"2024-03-07T07:20:08.024551+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:08.024551+0000","last_undegraded":"2024-03-07T07:20:08.024551+0000","last_fullsized":"2024-03-07T07:20:08.024551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039391+0000","last_change":"2024-03-07T07:20:08.035791+0000","last_active":"2024-03-07T07:20:08.039391+0000","last_peered":"2024-03-07T07:20:08.039391+0000","last_clean":"2024-03-07T07:20:08.039391+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:08.039391+0000","last_undegraded":"2024-03-07T07:20:08.039391+0000","last_fullsized":"2024-03-07T07:20:08.039391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039242+0000","last_change":"2024-03-07T07:20:08.034016+0000","last_active":"2024-03-07T07:20:08.039242+0000","last_peered":"2024-03-07T07:20:08.039242+0000","last_clean":"2024-03-07T07:20:08.039242+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:08.039242+0000","last_undegraded":"2024-03-07T07:20:08.039242+0000","last_fullsized":"2024-03-07T07:20:08.039242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042382+0000","last_change":"2024-03-07T07:20:08.035489+0000","last_active":"2024-03-07T07:20:08.042382+0000","last_peered":"2024-03-07T07:20:08.042382+0000","last_clean":"2024-03-07T07:20:08.042382+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:08.042382+0000","last_undegraded":"2024-03-07T07:20:08.042382+0000","last_fullsized":"2024-03-07T07:20:08.042382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042584+0000","last_change":"2024-03-07T07:20:08.037898+0000","last_active":"2024-03-07T07:20:08.042584+0000","last_peered":"2024-03-07T07:20:08.042584+0000","last_clean":"2024-03-07T07:20:08.042584+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:08.042584+0000","last_undegraded":"2024-03-07T07:20:08.042584+0000","last_fullsized":"2024-03-07T07:20:08.042584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032417+0000","last_change":"2024-03-07T07:20:08.025458+0000","last_active":"2024-03-07T07:20:08.032417+0000","last_peered":"2024-03-07T07:20:08.032417+0000","last_clean":"2024-03-07T07:20:08.032417+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:08.032417+0000","last_undegraded":"2024-03-07T07:20:08.032417+0000","last_fullsized":"2024-03-07T07:20:08.032417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:11:39.841810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018155+0000","last_change":"2024-03-07T07:20:08.040577+0000","last_active":"2024-03-07T07:20:09.018155+0000","last_peered":"2024-03-07T07:20:09.018155+0000","last_clean":"2024-03-07T07:20:09.018155+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:09.018155+0000","last_undegraded":"2024-03-07T07:20:09.018155+0000","last_fullsized":"2024-03-07T07:20:09.018155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018038+0000","last_change":"2024-03-07T07:20:08.039803+0000","last_active":"2024-03-07T07:20:09.018038+0000","last_peered":"2024-03-07T07:20:09.018038+0000","last_clean":"2024-03-07T07:20:09.018038+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:09.018038+0000","last_undegraded":"2024-03-07T07:20:09.018038+0000","last_fullsized":"2024-03-07T07:20:09.018038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015233+0000","last_change":"2024-03-07T07:20:08.031321+0000","last_active":"2024-03-07T07:20:09.015233+0000","last_peered":"2024-03-07T07:20:09.015233+0000","last_clean":"2024-03-07T07:20:09.015233+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:09.015233+0000","last_undegraded":"2024-03-07T07:20:09.015233+0000","last_fullsized":"2024-03-07T07:20:09.015233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.028156+0000","last_change":"2024-03-07T07:20:08.024172+0000","last_active":"2024-03-07T07:20:08.028156+0000","last_peered":"2024-03-07T07:20:08.028156+0000","last_clean":"2024-03-07T07:20:08.028156+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:08.028156+0000","last_undegraded":"2024-03-07T07:20:08.028156+0000","last_fullsized":"2024-03-07T07:20:08.028156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040820+0000","last_change":"2024-03-07T07:20:08.033067+0000","last_active":"2024-03-07T07:20:08.040820+0000","last_peered":"2024-03-07T07:20:08.040820+0000","last_clean":"2024-03-07T07:20:08.040820+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:08.040820+0000","last_undegraded":"2024-03-07T07:20:08.040820+0000","last_fullsized":"2024-03-07T07:20:08.040820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.035306+0000","last_change":"2024-03-07T07:20:08.031519+0000","last_active":"2024-03-07T07:20:08.035306+0000","last_peered":"2024-03-07T07:20:08.035306+0000","last_clean":"2024-03-07T07:20:08.035306+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:08.035306+0000","last_undegraded":"2024-03-07T07:20:08.035306+0000","last_fullsized":"2024-03-07T07:20:08.035306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.045921+0000","last_change":"2024-03-07T07:20:08.043585+0000","last_active":"2024-03-07T07:20:08.045921+0000","last_peered":"2024-03-07T07:20:08.045921+0000","last_clean":"2024-03-07T07:20:08.045921+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:08.045921+0000","last_undegraded":"2024-03-07T07:20:08.045921+0000","last_fullsized":"2024-03-07T07:20:08.045921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.024332+0000","last_change":"2024-03-07T07:20:08.021978+0000","last_active":"2024-03-07T07:20:08.024332+0000","last_peered":"2024-03-07T07:20:08.024332+0000","last_clean":"2024-03-07T07:20:08.024332+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:08.024332+0000","last_undegraded":"2024-03-07T07:20:08.024332+0000","last_fullsized":"2024-03-07T07:20:08.024332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018178+0000","last_change":"2024-03-07T07:20:08.040950+0000","last_active":"2024-03-07T07:20:09.018178+0000","last_peered":"2024-03-07T07:20:09.018178+0000","last_clean":"2024-03-07T07:20:09.018178+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:09.018178+0000","last_undegraded":"2024-03-07T07:20:09.018178+0000","last_fullsized":"2024-03-07T07:20:09.018178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.037637+0000","last_change":"2024-03-07T07:20:08.031636+0000","last_active":"2024-03-07T07:20:08.037637+0000","last_peered":"2024-03-07T07:20:08.037637+0000","last_clean":"2024-03-07T07:20:08.037637+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:08.037637+0000","last_undegraded":"2024-03-07T07:20:08.037637+0000","last_fullsized":"2024-03-07T07:20:08.037637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017643+0000","last_change":"2024-03-07T07:20:08.023430+0000","last_active":"2024-03-07T07:20:09.017643+0000","last_peered":"2024-03-07T07:20:09.017643+0000","last_clean":"2024-03-07T07:20:09.017643+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:09.017643+0000","last_undegraded":"2024-03-07T07:20:09.017643+0000","last_fullsized":"2024-03-07T07:20:09.017643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:29.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039333+0000","last_change":"2024-03-07T07:20:08.033808+0000","last_active":"2024-03-07T07:20:08.039333+0000","last_peered":"2024-03-07T07:20:08.039333+0000","last_clean":"2024-03-07T07:20:08.039333+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:08.039333+0000","last_undegraded":"2024-03-07T07:20:08.039333+0000","last_fullsized":"2024-03-07T07:20:08.039333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015411+0000","last_change":"2024-03-07T07:20:08.030548+0000","last_active":"2024-03-07T07:20:09.015411+0000","last_peered":"2024-03-07T07:20:09.015411+0000","last_clean":"2024-03-07T07:20:09.015411+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:09.015411+0000","last_undegraded":"2024-03-07T07:20:09.015411+0000","last_fullsized":"2024-03-07T07:20:09.015411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018115+0000","last_change":"2024-03-07T07:20:07.025352+0000","last_active":"2024-03-07T07:20:09.018115+0000","last_peered":"2024-03-07T07:20:09.018115+0000","last_clean":"2024-03-07T07:20:09.018115+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:09.018115+0000","last_undegraded":"2024-03-07T07:20:09.018115+0000","last_fullsized":"2024-03-07T07:20:09.018115+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018004+0000","last_change":"2024-03-07T07:20:08.033169+0000","last_active":"2024-03-07T07:20:09.018004+0000","last_peered":"2024-03-07T07:20:09.018004+0000","last_clean":"2024-03-07T07:20:09.018004+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:09.018004+0000","last_undegraded":"2024-03-07T07:20:09.018004+0000","last_fullsized":"2024-03-07T07:20:09.018004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015349+0000","last_change":"2024-03-07T07:20:07.025175+0000","last_active":"2024-03-07T07:20:09.015349+0000","last_peered":"2024-03-07T07:20:09.015349+0000","last_clean":"2024-03-07T07:20:09.015349+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:09.015349+0000","last_undegraded":"2024-03-07T07:20:09.015349+0000","last_fullsized":"2024-03-07T07:20:09.015349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:23.151665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018559+0000","last_change":"2024-03-07T07:20:08.040222+0000","last_active":"2024-03-07T07:20:09.018559+0000","last_peered":"2024-03-07T07:20:09.018559+0000","last_clean":"2024-03-07T07:20:09.018559+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:09.018559+0000","last_undegraded":"2024-03-07T07:20:09.018559+0000","last_fullsized":"2024-03-07T07:20:09.018559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043042+0000","last_change":"2024-03-07T07:20:07.033158+0000","last_active":"2024-03-07T07:20:08.043042+0000","last_peered":"2024-03-07T07:20:08.043042+0000","last_clean":"2024-03-07T07:20:08.043042+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:08.043042+0000","last_undegraded":"2024-03-07T07:20:08.043042+0000","last_fullsized":"2024-03-07T07:20:08.043042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:30.672111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.045905+0000","last_change":"2024-03-07T07:20:08.043374+0000","last_active":"2024-03-07T07:20:08.045905+0000","last_peered":"2024-03-07T07:20:08.045905+0000","last_clean":"2024-03-07T07:20:08.045905+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:08.045905+0000","last_undegraded":"2024-03-07T07:20:08.045905+0000","last_fullsized":"2024-03-07T07:20:08.045905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040909+0000","last_change":"2024-03-07T07:20:07.028373+0000","last_active":"2024-03-07T07:20:08.040909+0000","last_peered":"2024-03-07T07:20:08.040909+0000","last_clean":"2024-03-07T07:20:08.040909+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:08.040909+0000","last_undegraded":"2024-03-07T07:20:08.040909+0000","last_fullsized":"2024-03-07T07:20:08.040909+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:51.229977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039191+0000","last_change":"2024-03-07T07:20:08.033882+0000","last_active":"2024-03-07T07:20:08.039191+0000","last_peered":"2024-03-07T07:20:08.039191+0000","last_clean":"2024-03-07T07:20:08.039191+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:08.039191+0000","last_undegraded":"2024-03-07T07:20:08.039191+0000","last_fullsized":"2024-03-07T07:20:08.039191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018255+0000","last_change":"2024-03-07T07:20:07.028711+0000","last_active":"2024-03-07T07:20:09.018255+0000","last_peered":"2024-03-07T07:20:09.018255+0000","last_clean":"2024-03-07T07:20:09.018255+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:09.018255+0000","last_undegraded":"2024-03-07T07:20:09.018255+0000","last_fullsized":"2024-03-07T07:20:09.018255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:46.879544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038596+0000","last_change":"2024-03-07T07:20:08.030098+0000","last_active":"2024-03-07T07:20:08.038596+0000","last_peered":"2024-03-07T07:20:08.038596+0000","last_clean":"2024-03-07T07:20:08.038596+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:08.038596+0000","last_undegraded":"2024-03-07T07:20:08.038596+0000","last_fullsized":"2024-03-07T07:20:08.038596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018003+0000","last_change":"2024-03-07T07:20:07.029834+0000","last_active":"2024-03-07T07:20:09.018003+0000","last_peered":"2024-03-07T07:20:09.018003+0000","last_clean":"2024-03-07T07:20:09.018003+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:09.018003+0000","last_undegraded":"2024-03-07T07:20:09.018003+0000","last_fullsized":"2024-03-07T07:20:09.018003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:55:49.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018248+0000","last_change":"2024-03-07T07:20:08.028053+0000","last_active":"2024-03-07T07:20:09.018248+0000","last_peered":"2024-03-07T07:20:09.018248+0000","last_clean":"2024-03-07T07:20:09.018248+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:09.018248+0000","last_undegraded":"2024-03-07T07:20:09.018248+0000","last_fullsized":"2024-03-07T07:20:09.018248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.016495+0000","last_change":"2024-03-07T07:20:07.033411+0000","last_active":"2024-03-07T07:20:08.016495+0000","last_peered":"2024-03-07T07:20:08.016495+0000","last_clean":"2024-03-07T07:20:08.016495+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:08.016495+0000","last_undegraded":"2024-03-07T07:20:08.016495+0000","last_fullsized":"2024-03-07T07:20:08.016495+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:15.200144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040367+0000","last_change":"2024-03-07T07:20:08.032643+0000","last_active":"2024-03-07T07:20:08.040367+0000","last_peered":"2024-03-07T07:20:08.040367+0000","last_clean":"2024-03-07T07:20:08.040367+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:08.040367+0000","last_undegraded":"2024-03-07T07:20:08.040367+0000","last_fullsized":"2024-03-07T07:20:08.040367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018364+0000","last_change":"2024-03-07T07:20:07.027022+0000","last_active":"2024-03-07T07:20:09.018364+0000","last_peered":"2024-03-07T07:20:09.018364+0000","last_clean":"2024-03-07T07:20:09.018364+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:09.018364+0000","last_undegraded":"2024-03-07T07:20:09.018364+0000","last_fullsized":"2024-03-07T07:20:09.018364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:35.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032915+0000","last_change":"2024-03-07T07:20:08.028592+0000","last_active":"2024-03-07T07:20:08.032915+0000","last_peered":"2024-03-07T07:20:08.032915+0000","last_clean":"2024-03-07T07:20:08.032915+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:08.032915+0000","last_undegraded":"2024-03-07T07:20:08.032915+0000","last_fullsized":"2024-03-07T07:20:08.032915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.017522+0000","last_change":"2024-03-07T07:20:07.030367+0000","last_active":"2024-03-07T07:20:08.017522+0000","last_peered":"2024-03-07T07:20:08.017522+0000","last_clean":"2024-03-07T07:20:08.017522+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:08.017522+0000","last_undegraded":"2024-03-07T07:20:08.017522+0000","last_fullsized":"2024-03-07T07:20:08.017522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:12.128114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041454+0000","last_change":"2024-03-07T07:20:08.034949+0000","last_active":"2024-03-07T07:20:08.041454+0000","last_peered":"2024-03-07T07:20:08.041454+0000","last_clean":"2024-03-07T07:20:08.041454+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:08.041454+0000","last_undegraded":"2024-03-07T07:20:08.041454+0000","last_fullsized":"2024-03-07T07:20:08.041454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015409+0000","last_change":"2024-03-07T07:20:07.030049+0000","last_active":"2024-03-07T07:20:09.015409+0000","last_peered":"2024-03-07T07:20:09.015409+0000","last_clean":"2024-03-07T07:20:09.015409+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:09.015409+0000","last_undegraded":"2024-03-07T07:20:09.015409+0000","last_fullsized":"2024-03-07T07:20:09.015409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.033701+0000","last_change":"2024-03-07T07:20:08.030884+0000","last_active":"2024-03-07T07:20:08.033701+0000","last_peered":"2024-03-07T07:20:08.033701+0000","last_clean":"2024-03-07T07:20:08.033701+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:08.033701+0000","last_undegraded":"2024-03-07T07:20:08.033701+0000","last_fullsized":"2024-03-07T07:20:08.033701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034520+0000","last_change":"2024-03-07T07:20:07.028526+0000","last_active":"2024-03-07T07:20:08.034520+0000","last_peered":"2024-03-07T07:20:08.034520+0000","last_clean":"2024-03-07T07:20:08.034520+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:08.034520+0000","last_undegraded":"2024-03-07T07:20:08.034520+0000","last_fullsized":"2024-03-07T07:20:08.034520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041419+0000","last_change":"2024-03-07T07:20:08.038594+0000","last_active":"2024-03-07T07:20:08.041419+0000","last_peered":"2024-03-07T07:20:08.041419+0000","last_clean":"2024-03-07T07:20:08.041419+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:08.041419+0000","last_undegraded":"2024-03-07T07:20:08.041419+0000","last_fullsized":"2024-03-07T07:20:08.041419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.048918+0000","last_change":"2024-03-07T07:20:07.035824+0000","last_active":"2024-03-07T07:20:08.048918+0000","last_peered":"2024-03-07T07:20:08.048918+0000","last_clean":"2024-03-07T07:20:08.048918+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:08.048918+0000","last_undegraded":"2024-03-07T07:20:08.048918+0000","last_fullsized":"2024-03-07T07:20:08.048918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043867+0000","last_change":"2024-03-07T07:20:08.034611+0000","last_active":"2024-03-07T07:20:08.043867+0000","last_peered":"2024-03-07T07:20:08.043867+0000","last_clean":"2024-03-07T07:20:08.043867+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:08.043867+0000","last_undegraded":"2024-03-07T07:20:08.043867+0000","last_fullsized":"2024-03-07T07:20:08.043867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018274+0000","last_change":"2024-03-07T07:20:07.025705+0000","last_active":"2024-03-07T07:20:09.018274+0000","last_peered":"2024-03-07T07:20:09.018274+0000","last_clean":"2024-03-07T07:20:09.018274+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:09.018274+0000","last_undegraded":"2024-03-07T07:20:09.018274+0000","last_fullsized":"2024-03-07T07:20:09.018274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042617+0000","last_change":"2024-03-07T07:20:08.037145+0000","last_active":"2024-03-07T07:20:08.042617+0000","last_peered":"2024-03-07T07:20:08.042617+0000","last_clean":"2024-03-07T07:20:08.042617+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:08.042617+0000","last_undegraded":"2024-03-07T07:20:08.042617+0000","last_fullsized":"2024-03-07T07:20:08.042617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.026553+0000","last_change":"2024-03-07T07:20:07.026780+0000","last_active":"2024-03-07T07:20:08.026553+0000","last_peered":"2024-03-07T07:20:08.026553+0000","last_clean":"2024-03-07T07:20:08.026553+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:08.026553+0000","last_undegraded":"2024-03-07T07:20:08.026553+0000","last_fullsized":"2024-03-07T07:20:08.026553+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039355+0000","last_change":"2024-03-07T07:20:08.033869+0000","last_active":"2024-03-07T07:20:08.039355+0000","last_peered":"2024-03-07T07:20:08.039355+0000","last_clean":"2024-03-07T07:20:08.039355+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:08.039355+0000","last_undegraded":"2024-03-07T07:20:08.039355+0000","last_fullsized":"2024-03-07T07:20:08.039355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.017152+0000","last_change":"2024-03-07T07:20:07.032636+0000","last_active":"2024-03-07T07:20:08.017152+0000","last_peered":"2024-03-07T07:20:08.017152+0000","last_clean":"2024-03-07T07:20:08.017152+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:08.017152+0000","last_undegraded":"2024-03-07T07:20:08.017152+0000","last_fullsized":"2024-03-07T07:20:08.017152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017781+0000","last_change":"2024-03-07T07:20:08.036782+0000","last_active":"2024-03-07T07:20:09.017781+0000","last_peered":"2024-03-07T07:20:09.017781+0000","last_clean":"2024-03-07T07:20:09.017781+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:09.017781+0000","last_undegraded":"2024-03-07T07:20:09.017781+0000","last_fullsized":"2024-03-07T07:20:09.017781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:19.197402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042703+0000","last_change":"2024-03-07T07:20:07.033230+0000","last_active":"2024-03-07T07:20:08.042703+0000","last_peered":"2024-03-07T07:20:08.042703+0000","last_clean":"2024-03-07T07:20:08.042703+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:08.042703+0000","last_undegraded":"2024-03-07T07:20:08.042703+0000","last_fullsized":"2024-03-07T07:20:08.042703+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:42.777795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018625+0000","last_change":"2024-03-07T07:20:08.036934+0000","last_active":"2024-03-07T07:20:09.018625+0000","last_peered":"2024-03-07T07:20:09.018625+0000","last_clean":"2024-03-07T07:20:09.018625+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:09.018625+0000","last_undegraded":"2024-03-07T07:20:09.018625+0000","last_fullsized":"2024-03-07T07:20:09.018625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043377+0000","last_change":"2024-03-07T07:20:07.033018+0000","last_active":"2024-03-07T07:20:08.043377+0000","last_peered":"2024-03-07T07:20:08.043377+0000","last_clean":"2024-03-07T07:20:08.043377+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:08.043377+0000","last_undegraded":"2024-03-07T07:20:08.043377+0000","last_fullsized":"2024-03-07T07:20:08.043377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040666+0000","last_change":"2024-03-07T07:20:08.035720+0000","last_active":"2024-03-07T07:20:08.040666+0000","last_peered":"2024-03-07T07:20:08.040666+0000","last_clean":"2024-03-07T07:20:08.040666+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:08.040666+0000","last_undegraded":"2024-03-07T07:20:08.040666+0000","last_fullsized":"2024-03-07T07:20:08.040666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018650+0000","last_change":"2024-03-07T07:20:07.026560+0000","last_active":"2024-03-07T07:20:09.018650+0000","last_peered":"2024-03-07T07:20:09.018650+0000","last_clean":"2024-03-07T07:20:09.018650+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:09.018650+0000","last_undegraded":"2024-03-07T07:20:09.018650+0000","last_fullsized":"2024-03-07T07:20:09.018650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:52.614111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043805+0000","last_change":"2024-03-07T07:20:08.033852+0000","last_active":"2024-03-07T07:20:08.043805+0000","last_peered":"2024-03-07T07:20:08.043805+0000","last_clean":"2024-03-07T07:20:08.043805+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:08.043805+0000","last_undegraded":"2024-03-07T07:20:08.043805+0000","last_fullsized":"2024-03-07T07:20:08.043805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018510+0000","last_change":"2024-03-07T07:20:07.023668+0000","last_active":"2024-03-07T07:20:09.018510+0000","last_peered":"2024-03-07T07:20:09.018510+0000","last_clean":"2024-03-07T07:20:09.018510+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:09.018510+0000","last_undegraded":"2024-03-07T07:20:09.018510+0000","last_fullsized":"2024-03-07T07:20:09.018510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:07.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.033122+0000","last_change":"2024-03-07T07:20:08.030524+0000","last_active":"2024-03-07T07:20:08.033122+0000","last_peered":"2024-03-07T07:20:08.033122+0000","last_clean":"2024-03-07T07:20:08.033122+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:08.033122+0000","last_undegraded":"2024-03-07T07:20:08.033122+0000","last_fullsized":"2024-03-07T07:20:08.033122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:17:40.279427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018550+0000","last_change":"2024-03-07T07:20:07.029892+0000","last_active":"2024-03-07T07:20:09.018550+0000","last_peered":"2024-03-07T07:20:09.018550+0000","last_clean":"2024-03-07T07:20:09.018550+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:09.018550+0000","last_undegraded":"2024-03-07T07:20:09.018550+0000","last_fullsized":"2024-03-07T07:20:09.018550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:24.735113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034183+0000","last_change":"2024-03-07T07:20:08.032134+0000","last_active":"2024-03-07T07:20:08.034183+0000","last_peered":"2024-03-07T07:20:08.034183+0000","last_clean":"2024-03-07T07:20:08.034183+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:08.034183+0000","last_undegraded":"2024-03-07T07:20:08.034183+0000","last_fullsized":"2024-03-07T07:20:08.034183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:08.669181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015375+0000","last_change":"2024-03-07T07:20:07.028153+0000","last_active":"2024-03-07T07:20:09.015375+0000","last_peered":"2024-03-07T07:20:09.015375+0000","last_clean":"2024-03-07T07:20:09.015375+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:09.015375+0000","last_undegraded":"2024-03-07T07:20:09.015375+0000","last_fullsized":"2024-03-07T07:20:09.015375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034156+0000","last_change":"2024-03-07T07:20:08.032594+0000","last_active":"2024-03-07T07:20:08.034156+0000","last_peered":"2024-03-07T07:20:08.034156+0000","last_clean":"2024-03-07T07:20:08.034156+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:08.034156+0000","last_undegraded":"2024-03-07T07:20:08.034156+0000","last_fullsized":"2024-03-07T07:20:08.034156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:36.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018303+0000","last_change":"2024-03-07T07:20:07.028455+0000","last_active":"2024-03-07T07:20:09.018303+0000","last_peered":"2024-03-07T07:20:09.018303+0000","last_clean":"2024-03-07T07:20:09.018303+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:09.018303+0000","last_undegraded":"2024-03-07T07:20:09.018303+0000","last_fullsized":"2024-03-07T07:20:09.018303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.033039+0000","last_change":"2024-03-07T07:20:08.031299+0000","last_active":"2024-03-07T07:20:08.033039+0000","last_peered":"2024-03-07T07:20:08.033039+0000","last_clean":"2024-03-07T07:20:08.033039+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:08.033039+0000","last_undegraded":"2024-03-07T07:20:08.033039+0000","last_fullsized":"2024-03-07T07:20:08.033039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:37:06.166772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017852+0000","last_change":"2024-03-07T07:20:07.028369+0000","last_active":"2024-03-07T07:20:09.017852+0000","last_peered":"2024-03-07T07:20:09.017852+0000","last_clean":"2024-03-07T07:20:09.017852+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:09.017852+0000","last_undegraded":"2024-03-07T07:20:09.017852+0000","last_fullsized":"2024-03-07T07:20:09.017852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038359+0000","last_change":"2024-03-07T07:20:08.033176+0000","last_active":"2024-03-07T07:20:08.038359+0000","last_peered":"2024-03-07T07:20:08.038359+0000","last_clean":"2024-03-07T07:20:08.038359+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:08.038359+0000","last_undegraded":"2024-03-07T07:20:08.038359+0000","last_fullsized":"2024-03-07T07:20:08.038359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.017914+0000","last_change":"2024-03-07T07:20:07.029244+0000","last_active":"2024-03-07T07:20:08.017914+0000","last_peered":"2024-03-07T07:20:08.017914+0000","last_clean":"2024-03-07T07:20:08.017914+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:08.017914+0000","last_undegraded":"2024-03-07T07:20:08.017914+0000","last_fullsized":"2024-03-07T07:20:08.017914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032747+0000","last_change":"2024-03-07T07:20:08.026414+0000","last_active":"2024-03-07T07:20:08.032747+0000","last_peered":"2024-03-07T07:20:08.032747+0000","last_clean":"2024-03-07T07:20:08.032747+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:08.032747+0000","last_undegraded":"2024-03-07T07:20:08.032747+0000","last_fullsized":"2024-03-07T07:20:08.032747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:26:04.002320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041613+0000","last_change":"2024-03-07T07:20:07.027164+0000","last_active":"2024-03-07T07:20:08.041613+0000","last_peered":"2024-03-07T07:20:08.041613+0000","last_clean":"2024-03-07T07:20:08.041613+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:08.041613+0000","last_undegraded":"2024-03-07T07:20:08.041613+0000","last_fullsized":"2024-03-07T07:20:08.041613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040644+0000","last_change":"2024-03-07T07:20:08.035488+0000","last_active":"2024-03-07T07:20:08.040644+0000","last_peered":"2024-03-07T07:20:08.040644+0000","last_clean":"2024-03-07T07:20:08.040644+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:08.040644+0000","last_undegraded":"2024-03-07T07:20:08.040644+0000","last_fullsized":"2024-03-07T07:20:08.040644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015456+0000","last_change":"2024-03-07T07:20:07.027534+0000","last_active":"2024-03-07T07:20:09.015456+0000","last_peered":"2024-03-07T07:20:09.015456+0000","last_clean":"2024-03-07T07:20:09.015456+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:09.015456+0000","last_undegraded":"2024-03-07T07:20:09.015456+0000","last_fullsized":"2024-03-07T07:20:09.015456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015303+0000","last_change":"2024-03-07T07:20:08.036269+0000","last_active":"2024-03-07T07:20:09.015303+0000","last_peered":"2024-03-07T07:20:09.015303+0000","last_clean":"2024-03-07T07:20:09.015303+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:09.015303+0000","last_undegraded":"2024-03-07T07:20:09.015303+0000","last_fullsized":"2024-03-07T07:20:09.015303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.025886+0000","last_change":"2024-03-07T07:20:07.023056+0000","last_active":"2024-03-07T07:20:08.025886+0000","last_peered":"2024-03-07T07:20:08.025886+0000","last_clean":"2024-03-07T07:20:08.025886+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:08.025886+0000","last_undegraded":"2024-03-07T07:20:08.025886+0000","last_fullsized":"2024-03-07T07:20:08.025886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018197+0000","last_change":"2024-03-07T07:20:08.023363+0000","last_active":"2024-03-07T07:20:09.018197+0000","last_peered":"2024-03-07T07:20:09.018197+0000","last_clean":"2024-03-07T07:20:09.018197+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:09.018197+0000","last_undegraded":"2024-03-07T07:20:09.018197+0000","last_fullsized":"2024-03-07T07:20:09.018197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015478+0000","last_change":"2024-03-07T07:20:07.024131+0000","last_active":"2024-03-07T07:20:09.015478+0000","last_peered":"2024-03-07T07:20:09.015478+0000","last_clean":"2024-03-07T07:20:09.015478+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:09.015478+0000","last_undegraded":"2024-03-07T07:20:09.015478+0000","last_fullsized":"2024-03-07T07:20:09.015478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015390+0000","last_change":"2024-03-07T07:20:08.030007+0000","last_active":"2024-03-07T07:20:09.015390+0000","last_peered":"2024-03-07T07:20:09.015390+0000","last_clean":"2024-03-07T07:20:09.015390+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:09.015390+0000","last_undegraded":"2024-03-07T07:20:09.015390+0000","last_fullsized":"2024-03-07T07:20:09.015390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.026687+0000","last_change":"2024-03-07T07:20:07.025544+0000","last_active":"2024-03-07T07:20:08.026687+0000","last_peered":"2024-03-07T07:20:08.026687+0000","last_clean":"2024-03-07T07:20:08.026687+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:08.026687+0000","last_undegraded":"2024-03-07T07:20:08.026687+0000","last_fullsized":"2024-03-07T07:20:08.026687+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:18.193822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041157+0000","last_change":"2024-03-07T07:20:08.035274+0000","last_active":"2024-03-07T07:20:08.041157+0000","last_peered":"2024-03-07T07:20:08.041157+0000","last_clean":"2024-03-07T07:20:08.041157+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:08.041157+0000","last_undegraded":"2024-03-07T07:20:08.041157+0000","last_fullsized":"2024-03-07T07:20:08.041157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017946+0000","last_change":"2024-03-07T07:20:07.023385+0000","last_active":"2024-03-07T07:20:09.017946+0000","last_peered":"2024-03-07T07:20:09.017946+0000","last_clean":"2024-03-07T07:20:09.017946+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:09.017946+0000","last_undegraded":"2024-03-07T07:20:09.017946+0000","last_fullsized":"2024-03-07T07:20:09.017946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017838+0000","last_change":"2024-03-07T07:20:08.040036+0000","last_active":"2024-03-07T07:20:09.017838+0000","last_peered":"2024-03-07T07:20:09.017838+0000","last_clean":"2024-03-07T07:20:09.017838+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:09.017838+0000","last_undegraded":"2024-03-07T07:20:09.017838+0000","last_fullsized":"2024-03-07T07:20:09.017838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.016780+0000","last_change":"2024-03-07T07:20:07.036824+0000","last_active":"2024-03-07T07:20:08.016780+0000","last_peered":"2024-03-07T07:20:08.016780+0000","last_clean":"2024-03-07T07:20:08.016780+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:08.016780+0000","last_undegraded":"2024-03-07T07:20:08.016780+0000","last_fullsized":"2024-03-07T07:20:08.016780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:20.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039292+0000","last_change":"2024-03-07T07:20:08.034942+0000","last_active":"2024-03-07T07:20:08.039292+0000","last_peered":"2024-03-07T07:20:08.039292+0000","last_clean":"2024-03-07T07:20:08.039292+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:08.039292+0000","last_undegraded":"2024-03-07T07:20:08.039292+0000","last_fullsized":"2024-03-07T07:20:08.039292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017654+0000","last_change":"2024-03-07T07:20:07.023606+0000","last_active":"2024-03-07T07:20:09.017654+0000","last_peered":"2024-03-07T07:20:09.017654+0000","last_clean":"2024-03-07T07:20:09.017654+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:09.017654+0000","last_undegraded":"2024-03-07T07:20:09.017654+0000","last_fullsized":"2024-03-07T07:20:09.017654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.026731+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:20:08.026731+0000","last_peered":"2024-03-07T07:20:08.026731+0000","last_clean":"2024-03-07T07:20:08.026731+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:08.026731+0000","last_undegraded":"2024-03-07T07:20:08.026731+0000","last_fullsized":"2024-03-07T07:20:08.026731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":730774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":730684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:20:11.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:20:11.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.186+0000 7fd04db81640 1 Processor -- start 2024-03-07T07:20:11.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.186+0000 7fd04db81640 1 -- start start 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd04db81640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd048162750 0x7fd04806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd04db81640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd04806be80 0x7fd0480cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd04db81640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd048161cb0 con 0x7fd0480d1870 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd04db81640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0480ce620 con 0x7fd048162750 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd04db81640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04806c2f0 con 0x7fd04806be80 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd046ffd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd048162750 0x7fd04806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd0477fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fd0480d1870 0x7fd0480cd910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60152/0 (socket says 172.21.15.64:60152) 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd0477fe640 1 -- 172.21.15.64:0/2776281089 learned_addr learned my addr 172.21.15.64:0/2776281089 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd047fff640 1 --2- 172.21.15.64:0/2776281089 >> v2:172.21.15.120:3300/0 conn(0x7fd04806be80 0x7fd0480cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd046ffd640 1 -- 172.21.15.64:0/2776281089 >> v2:172.21.15.120:3300/0 conn(0x7fd04806be80 msgr2=0x7fd0480cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:11.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd046ffd640 1 --2- 172.21.15.64:0/2776281089 >> v2:172.21.15.120:3300/0 conn(0x7fd04806be80 0x7fd0480cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd046ffd640 1 -- 172.21.15.64:0/2776281089 >> v1:172.21.15.64:6789/0 conn(0x7fd0480d1870 legacy=0x7fd0480cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd0467fc640 1 -- 172.21.15.64:0/2776281089 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173853487 0 0) 0x7fd048161cb0 con 0x7fd0480d1870 2024-03-07T07:20:11.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd046ffd640 1 -- 172.21.15.64:0/2776281089 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480ce340 con 0x7fd048162750 2024-03-07T07:20:11.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.187+0000 7fd047fff640 1 --2- 172.21.15.64:0/2776281089 >> v2:172.21.15.120:3300/0 conn(0x7fd04806be80 0x7fd0480cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:11.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.188+0000 7fd046ffd640 1 --2- 172.21.15.64:0/2776281089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd048162750 0x7fd04806b940 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd048072710 tx=0x7fd03c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1452cefd3fdbe447 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:11.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.188+0000 7fd0467fc640 1 -- 172.21.15.64:0/2776281089 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c035020 con 0x7fd048162750 2024-03-07T07:20:11.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.188+0000 7fd0467fc640 1 -- 172.21.15.64:0/2776281089 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c031830 con 0x7fd048162750 2024-03-07T07:20:11.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.188+0000 7fd0467fc640 1 -- 172.21.15.64:0/2776281089 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c031c00 con 0x7fd048162750 2024-03-07T07:20:11.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 -- 172.21.15.64:0/2776281089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd048162750 msgr2=0x7fd04806b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 --2- 172.21.15.64:0/2776281089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd048162750 0x7fd04806b940 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd048072710 tx=0x7fd03c02b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 -- 172.21.15.64:0/2776281089 shutdown_connections 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 --2- 172.21.15.64:0/2776281089 >> v2:172.21.15.120:3300/0 conn(0x7fd04806be80 0x7fd0480cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 --2- 172.21.15.64:0/2776281089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd048162750 0x7fd04806b940 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 -- 172.21.15.64:0/2776281089 >> 172.21.15.64:0/2776281089 conn(0x7fd048066e50 msgr2=0x7fd048067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 -- 172.21.15.64:0/2776281089 shutdown_connections 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.189+0000 7fd04db81640 1 -- 172.21.15.64:0/2776281089 wait complete. 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.190+0000 7fd04db81640 1 Processor -- start 2024-03-07T07:20:11.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.190+0000 7fd04db81640 1 -- start start 2024-03-07T07:20:11.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd04db81640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd04806be80 0x7fd048153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd04db81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0480d1870 0x7fd048156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd04db81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd048162750 0x7fd048156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd04db81640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480ce020 con 0x7fd0480d1870 2024-03-07T07:20:11.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd04db81640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048162230 con 0x7fd048162750 2024-03-07T07:20:11.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd04db81640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04815f2c0 con 0x7fd04806be80 2024-03-07T07:20:11.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd046ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0480d1870 0x7fd048156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd046ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0480d1870 0x7fd048156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60316/0 (socket says 172.21.15.64:60316) 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd046ffd640 1 -- 172.21.15.64:0/3153763270 learned_addr learned my addr 172.21.15.64:0/3153763270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd0477fe640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd04806be80 0x7fd048153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd047fff640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd048162750 0x7fd048156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd046ffd640 1 -- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd04806be80 msgr2=0x7fd048153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.191+0000 7fd046ffd640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd04806be80 0x7fd048153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd046ffd640 1 -- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd048162750 msgr2=0x7fd048156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd046ffd640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd048162750 0x7fd048156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd046ffd640 1 -- 172.21.15.64:0/3153763270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0481f7260 con 0x7fd0480d1870 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd0477fe640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd04806be80 0x7fd048153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd047fff640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd048162750 0x7fd048156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd046ffd640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0480d1870 0x7fd048156140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009890 tx=0x7fd03c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd02ffff640 1 -- 172.21.15.64:0/3153763270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c040070 con 0x7fd0480d1870 2024-03-07T07:20:11.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd02ffff640 1 -- 172.21.15.64:0/3153763270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c039a20 con 0x7fd0480d1870 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0481f7540 con 0x7fd0480d1870 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd02ffff640 1 -- 172.21.15.64:0/3153763270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c035040 con 0x7fd0480d1870 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.192+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0481f8020 con 0x7fd0480d1870 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.193+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd048062c90 con 0x7fd0480d1870 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.193+0000 7fd02ffff640 1 -- 172.21.15.64:0/3153763270 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd03c038020 con 0x7fd0480d1870 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.194+0000 7fd02ffff640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd03406b4d0 0x7fd03406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.194+0000 7fd02ffff640 1 -- 172.21.15.64:0/3153763270 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd03c0abd40 con 0x7fd0480d1870 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.194+0000 7fd0477fe640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd03406b4d0 0x7fd03406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.194+0000 7fd0477fe640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd03406b4d0 0x7fd03406d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd038007fd0 tx=0x7fd038009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:11.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.198+0000 7fd02ffff640 1 -- 172.21.15.64:0/3153763270 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd03c04b340 con 0x7fd0480d1870 2024-03-07T07:20:11.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.376+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd048074310 con 0x7fd03406b4d0 2024-03-07T07:20:11.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.382+0000 7fd02ffff640 1 -- 172.21.15.64:0/3153763270 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251900 (secure 0 0 0) 0x7fd048074310 con 0x7fd03406b4d0 2024-03-07T07:20:11.385 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:11.386 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd03406b4d0 msgr2=0x7fd03406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd03406b4d0 0x7fd03406d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd038007fd0 tx=0x7fd038009040 comp rx=0 tx=0).stop 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0480d1870 msgr2=0x7fd048156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0480d1870 0x7fd048156140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009890 tx=0x7fd03c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 shutdown_connections 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd03406b4d0 0x7fd03406d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0480d1870 0x7fd048156140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd048162750 0x7fd048156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 --2- 172.21.15.64:0/3153763270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd04806be80 0x7fd048153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 >> 172.21.15.64:0/3153763270 conn(0x7fd048066e50 msgr2=0x7fd0480714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:11.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 shutdown_connections 2024-03-07T07:20:11.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.385+0000 7fd04db81640 1 -- 172.21.15.64:0/3153763270 wait complete. 2024-03-07T07:20:11.396 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-03-07T07:20:10.189949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220208,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754512,"statfs":{"total":773094113280,"available":772868620288,"internally_reserved":0,"allocated":4136960,"data_stored":2308084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.182899"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018002+0000","last_change":"2024-03-07T07:20:08.025425+0000","last_active":"2024-03-07T07:20:09.018002+0000","last_peered":"2024-03-07T07:20:09.018002+0000","last_clean":"2024-03-07T07:20:09.018002+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:09.018002+0000","last_undegraded":"2024-03-07T07:20:09.018002+0000","last_fullsized":"2024-03-07T07:20:09.018002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042407+0000","last_change":"2024-03-07T07:20:08.036714+0000","last_active":"2024-03-07T07:20:08.042407+0000","last_peered":"2024-03-07T07:20:08.042407+0000","last_clean":"2024-03-07T07:20:08.042407+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:08.042407+0000","last_undegraded":"2024-03-07T07:20:08.042407+0000","last_fullsized":"2024-03-07T07:20:08.042407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041235+0000","last_change":"2024-03-07T07:20:08.037973+0000","last_active":"2024-03-07T07:20:08.041235+0000","last_peered":"2024-03-07T07:20:08.041235+0000","last_clean":"2024-03-07T07:20:08.041235+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:08.041235+0000","last_undegraded":"2024-03-07T07:20:08.041235+0000","last_fullsized":"2024-03-07T07:20:08.041235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.045882+0000","last_change":"2024-03-07T07:20:08.042993+0000","last_active":"2024-03-07T07:20:08.045882+0000","last_peered":"2024-03-07T07:20:08.045882+0000","last_clean":"2024-03-07T07:20:08.045882+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:08.045882+0000","last_undegraded":"2024-03-07T07:20:08.045882+0000","last_fullsized":"2024-03-07T07:20:08.045882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043121+0000","last_change":"2024-03-07T07:20:08.037872+0000","last_active":"2024-03-07T07:20:08.043121+0000","last_peered":"2024-03-07T07:20:08.043121+0000","last_clean":"2024-03-07T07:20:08.043121+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:08.043121+0000","last_undegraded":"2024-03-07T07:20:08.043121+0000","last_fullsized":"2024-03-07T07:20:08.043121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:53.617691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032465+0000","last_change":"2024-03-07T07:20:08.028212+0000","last_active":"2024-03-07T07:20:08.032465+0000","last_peered":"2024-03-07T07:20:08.032465+0000","last_clean":"2024-03-07T07:20:08.032465+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:08.032465+0000","last_undegraded":"2024-03-07T07:20:08.032465+0000","last_fullsized":"2024-03-07T07:20:08.032465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038277+0000","last_change":"2024-03-07T07:20:08.032379+0000","last_active":"2024-03-07T07:20:08.038277+0000","last_peered":"2024-03-07T07:20:08.038277+0000","last_clean":"2024-03-07T07:20:08.038277+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:08.038277+0000","last_undegraded":"2024-03-07T07:20:08.038277+0000","last_fullsized":"2024-03-07T07:20:08.038277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039373+0000","last_change":"2024-03-07T07:20:08.035495+0000","last_active":"2024-03-07T07:20:08.039373+0000","last_peered":"2024-03-07T07:20:08.039373+0000","last_clean":"2024-03-07T07:20:08.039373+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:08.039373+0000","last_undegraded":"2024-03-07T07:20:08.039373+0000","last_fullsized":"2024-03-07T07:20:08.039373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018731+0000","last_change":"2024-03-07T07:20:08.034012+0000","last_active":"2024-03-07T07:20:09.018731+0000","last_peered":"2024-03-07T07:20:09.018731+0000","last_clean":"2024-03-07T07:20:09.018731+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:09.018731+0000","last_undegraded":"2024-03-07T07:20:09.018731+0000","last_fullsized":"2024-03-07T07:20:09.018731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.044040+0000","last_change":"2024-03-07T07:20:08.037279+0000","last_active":"2024-03-07T07:20:08.044040+0000","last_peered":"2024-03-07T07:20:08.044040+0000","last_clean":"2024-03-07T07:20:08.044040+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:08.044040+0000","last_undegraded":"2024-03-07T07:20:08.044040+0000","last_fullsized":"2024-03-07T07:20:08.044040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041719+0000","last_change":"2024-03-07T07:20:08.039682+0000","last_active":"2024-03-07T07:20:08.041719+0000","last_peered":"2024-03-07T07:20:08.041719+0000","last_clean":"2024-03-07T07:20:08.041719+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:08.041719+0000","last_undegraded":"2024-03-07T07:20:08.041719+0000","last_fullsized":"2024-03-07T07:20:08.041719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038659+0000","last_change":"2024-03-07T07:20:08.028305+0000","last_active":"2024-03-07T07:20:08.038659+0000","last_peered":"2024-03-07T07:20:08.038659+0000","last_clean":"2024-03-07T07:20:08.038659+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:08.038659+0000","last_undegraded":"2024-03-07T07:20:08.038659+0000","last_fullsized":"2024-03-07T07:20:08.038659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034862+0000","last_change":"2024-03-07T07:20:08.033602+0000","last_active":"2024-03-07T07:20:08.034862+0000","last_peered":"2024-03-07T07:20:08.034862+0000","last_clean":"2024-03-07T07:20:08.034862+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:08.034862+0000","last_undegraded":"2024-03-07T07:20:08.034862+0000","last_fullsized":"2024-03-07T07:20:08.034862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:25.738694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.030215+0000","last_change":"2024-03-07T07:20:08.022351+0000","last_active":"2024-03-07T07:20:08.030215+0000","last_peered":"2024-03-07T07:20:08.030215+0000","last_clean":"2024-03-07T07:20:08.030215+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:08.030215+0000","last_undegraded":"2024-03-07T07:20:08.030215+0000","last_fullsized":"2024-03-07T07:20:08.030215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.024551+0000","last_change":"2024-03-07T07:20:08.021148+0000","last_active":"2024-03-07T07:20:08.024551+0000","last_peered":"2024-03-07T07:20:08.024551+0000","last_clean":"2024-03-07T07:20:08.024551+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:08.024551+0000","last_undegraded":"2024-03-07T07:20:08.024551+0000","last_fullsized":"2024-03-07T07:20:08.024551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039391+0000","last_change":"2024-03-07T07:20:08.035791+0000","last_active":"2024-03-07T07:20:08.039391+0000","last_peered":"2024-03-07T07:20:08.039391+0000","last_clean":"2024-03-07T07:20:08.039391+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:08.039391+0000","last_undegraded":"2024-03-07T07:20:08.039391+0000","last_fullsized":"2024-03-07T07:20:08.039391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039242+0000","last_change":"2024-03-07T07:20:08.034016+0000","last_active":"2024-03-07T07:20:08.039242+0000","last_peered":"2024-03-07T07:20:08.039242+0000","last_clean":"2024-03-07T07:20:08.039242+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:08.039242+0000","last_undegraded":"2024-03-07T07:20:08.039242+0000","last_fullsized":"2024-03-07T07:20:08.039242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042382+0000","last_change":"2024-03-07T07:20:08.035489+0000","last_active":"2024-03-07T07:20:08.042382+0000","last_peered":"2024-03-07T07:20:08.042382+0000","last_clean":"2024-03-07T07:20:08.042382+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:08.042382+0000","last_undegraded":"2024-03-07T07:20:08.042382+0000","last_fullsized":"2024-03-07T07:20:08.042382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042584+0000","last_change":"2024-03-07T07:20:08.037898+0000","last_active":"2024-03-07T07:20:08.042584+0000","last_peered":"2024-03-07T07:20:08.042584+0000","last_clean":"2024-03-07T07:20:08.042584+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:08.042584+0000","last_undegraded":"2024-03-07T07:20:08.042584+0000","last_fullsized":"2024-03-07T07:20:08.042584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032417+0000","last_change":"2024-03-07T07:20:08.025458+0000","last_active":"2024-03-07T07:20:08.032417+0000","last_peered":"2024-03-07T07:20:08.032417+0000","last_clean":"2024-03-07T07:20:08.032417+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:08.032417+0000","last_undegraded":"2024-03-07T07:20:08.032417+0000","last_fullsized":"2024-03-07T07:20:08.032417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:11:39.841810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018155+0000","last_change":"2024-03-07T07:20:08.040577+0000","last_active":"2024-03-07T07:20:09.018155+0000","last_peered":"2024-03-07T07:20:09.018155+0000","last_clean":"2024-03-07T07:20:09.018155+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:09.018155+0000","last_undegraded":"2024-03-07T07:20:09.018155+0000","last_fullsized":"2024-03-07T07:20:09.018155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018038+0000","last_change":"2024-03-07T07:20:08.039803+0000","last_active":"2024-03-07T07:20:09.018038+0000","last_peered":"2024-03-07T07:20:09.018038+0000","last_clean":"2024-03-07T07:20:09.018038+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:09.018038+0000","last_undegraded":"2024-03-07T07:20:09.018038+0000","last_fullsized":"2024-03-07T07:20:09.018038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015233+0000","last_change":"2024-03-07T07:20:08.031321+0000","last_active":"2024-03-07T07:20:09.015233+0000","last_peered":"2024-03-07T07:20:09.015233+0000","last_clean":"2024-03-07T07:20:09.015233+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:09.015233+0000","last_undegraded":"2024-03-07T07:20:09.015233+0000","last_fullsized":"2024-03-07T07:20:09.015233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.028156+0000","last_change":"2024-03-07T07:20:08.024172+0000","last_active":"2024-03-07T07:20:08.028156+0000","last_peered":"2024-03-07T07:20:08.028156+0000","last_clean":"2024-03-07T07:20:08.028156+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:08.028156+0000","last_undegraded":"2024-03-07T07:20:08.028156+0000","last_fullsized":"2024-03-07T07:20:08.028156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040820+0000","last_change":"2024-03-07T07:20:08.033067+0000","last_active":"2024-03-07T07:20:08.040820+0000","last_peered":"2024-03-07T07:20:08.040820+0000","last_clean":"2024-03-07T07:20:08.040820+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:08.040820+0000","last_undegraded":"2024-03-07T07:20:08.040820+0000","last_fullsized":"2024-03-07T07:20:08.040820+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.035306+0000","last_change":"2024-03-07T07:20:08.031519+0000","last_active":"2024-03-07T07:20:08.035306+0000","last_peered":"2024-03-07T07:20:08.035306+0000","last_clean":"2024-03-07T07:20:08.035306+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:08.035306+0000","last_undegraded":"2024-03-07T07:20:08.035306+0000","last_fullsized":"2024-03-07T07:20:08.035306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.045921+0000","last_change":"2024-03-07T07:20:08.043585+0000","last_active":"2024-03-07T07:20:08.045921+0000","last_peered":"2024-03-07T07:20:08.045921+0000","last_clean":"2024-03-07T07:20:08.045921+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:08.045921+0000","last_undegraded":"2024-03-07T07:20:08.045921+0000","last_fullsized":"2024-03-07T07:20:08.045921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.024332+0000","last_change":"2024-03-07T07:20:08.021978+0000","last_active":"2024-03-07T07:20:08.024332+0000","last_peered":"2024-03-07T07:20:08.024332+0000","last_clean":"2024-03-07T07:20:08.024332+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:08.024332+0000","last_undegraded":"2024-03-07T07:20:08.024332+0000","last_fullsized":"2024-03-07T07:20:08.024332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018178+0000","last_change":"2024-03-07T07:20:08.040950+0000","last_active":"2024-03-07T07:20:09.018178+0000","last_peered":"2024-03-07T07:20:09.018178+0000","last_clean":"2024-03-07T07:20:09.018178+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:09.018178+0000","last_undegraded":"2024-03-07T07:20:09.018178+0000","last_fullsized":"2024-03-07T07:20:09.018178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.037637+0000","last_change":"2024-03-07T07:20:08.031636+0000","last_active":"2024-03-07T07:20:08.037637+0000","last_peered":"2024-03-07T07:20:08.037637+0000","last_clean":"2024-03-07T07:20:08.037637+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:08.037637+0000","last_undegraded":"2024-03-07T07:20:08.037637+0000","last_fullsized":"2024-03-07T07:20:08.037637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017643+0000","last_change":"2024-03-07T07:20:08.023430+0000","last_active":"2024-03-07T07:20:09.017643+0000","last_peered":"2024-03-07T07:20:09.017643+0000","last_clean":"2024-03-07T07:20:09.017643+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:09.017643+0000","last_undegraded":"2024-03-07T07:20:09.017643+0000","last_fullsized":"2024-03-07T07:20:09.017643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:29.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039333+0000","last_change":"2024-03-07T07:20:08.033808+0000","last_active":"2024-03-07T07:20:08.039333+0000","last_peered":"2024-03-07T07:20:08.039333+0000","last_clean":"2024-03-07T07:20:08.039333+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:08.039333+0000","last_undegraded":"2024-03-07T07:20:08.039333+0000","last_fullsized":"2024-03-07T07:20:08.039333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015411+0000","last_change":"2024-03-07T07:20:08.030548+0000","last_active":"2024-03-07T07:20:09.015411+0000","last_peered":"2024-03-07T07:20:09.015411+0000","last_clean":"2024-03-07T07:20:09.015411+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:09.015411+0000","last_undegraded":"2024-03-07T07:20:09.015411+0000","last_fullsized":"2024-03-07T07:20:09.015411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018115+0000","last_change":"2024-03-07T07:20:07.025352+0000","last_active":"2024-03-07T07:20:09.018115+0000","last_peered":"2024-03-07T07:20:09.018115+0000","last_clean":"2024-03-07T07:20:09.018115+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:09.018115+0000","last_undegraded":"2024-03-07T07:20:09.018115+0000","last_fullsized":"2024-03-07T07:20:09.018115+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018004+0000","last_change":"2024-03-07T07:20:08.033169+0000","last_active":"2024-03-07T07:20:09.018004+0000","last_peered":"2024-03-07T07:20:09.018004+0000","last_clean":"2024-03-07T07:20:09.018004+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:09.018004+0000","last_undegraded":"2024-03-07T07:20:09.018004+0000","last_fullsized":"2024-03-07T07:20:09.018004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015349+0000","last_change":"2024-03-07T07:20:07.025175+0000","last_active":"2024-03-07T07:20:09.015349+0000","last_peered":"2024-03-07T07:20:09.015349+0000","last_clean":"2024-03-07T07:20:09.015349+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:09.015349+0000","last_undegraded":"2024-03-07T07:20:09.015349+0000","last_fullsized":"2024-03-07T07:20:09.015349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:23.151665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018559+0000","last_change":"2024-03-07T07:20:08.040222+0000","last_active":"2024-03-07T07:20:09.018559+0000","last_peered":"2024-03-07T07:20:09.018559+0000","last_clean":"2024-03-07T07:20:09.018559+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:09.018559+0000","last_undegraded":"2024-03-07T07:20:09.018559+0000","last_fullsized":"2024-03-07T07:20:09.018559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043042+0000","last_change":"2024-03-07T07:20:07.033158+0000","last_active":"2024-03-07T07:20:08.043042+0000","last_peered":"2024-03-07T07:20:08.043042+0000","last_clean":"2024-03-07T07:20:08.043042+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:08.043042+0000","last_undegraded":"2024-03-07T07:20:08.043042+0000","last_fullsized":"2024-03-07T07:20:08.043042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:30.672111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.045905+0000","last_change":"2024-03-07T07:20:08.043374+0000","last_active":"2024-03-07T07:20:08.045905+0000","last_peered":"2024-03-07T07:20:08.045905+0000","last_clean":"2024-03-07T07:20:08.045905+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:08.045905+0000","last_undegraded":"2024-03-07T07:20:08.045905+0000","last_fullsized":"2024-03-07T07:20:08.045905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040909+0000","last_change":"2024-03-07T07:20:07.028373+0000","last_active":"2024-03-07T07:20:08.040909+0000","last_peered":"2024-03-07T07:20:08.040909+0000","last_clean":"2024-03-07T07:20:08.040909+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:08.040909+0000","last_undegraded":"2024-03-07T07:20:08.040909+0000","last_fullsized":"2024-03-07T07:20:08.040909+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:51.229977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039191+0000","last_change":"2024-03-07T07:20:08.033882+0000","last_active":"2024-03-07T07:20:08.039191+0000","last_peered":"2024-03-07T07:20:08.039191+0000","last_clean":"2024-03-07T07:20:08.039191+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:08.039191+0000","last_undegraded":"2024-03-07T07:20:08.039191+0000","last_fullsized":"2024-03-07T07:20:08.039191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018255+0000","last_change":"2024-03-07T07:20:07.028711+0000","last_active":"2024-03-07T07:20:09.018255+0000","last_peered":"2024-03-07T07:20:09.018255+0000","last_clean":"2024-03-07T07:20:09.018255+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:09.018255+0000","last_undegraded":"2024-03-07T07:20:09.018255+0000","last_fullsized":"2024-03-07T07:20:09.018255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:46.879544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038596+0000","last_change":"2024-03-07T07:20:08.030098+0000","last_active":"2024-03-07T07:20:08.038596+0000","last_peered":"2024-03-07T07:20:08.038596+0000","last_clean":"2024-03-07T07:20:08.038596+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:08.038596+0000","last_undegraded":"2024-03-07T07:20:08.038596+0000","last_fullsized":"2024-03-07T07:20:08.038596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018003+0000","last_change":"2024-03-07T07:20:07.029834+0000","last_active":"2024-03-07T07:20:09.018003+0000","last_peered":"2024-03-07T07:20:09.018003+0000","last_clean":"2024-03-07T07:20:09.018003+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:09.018003+0000","last_undegraded":"2024-03-07T07:20:09.018003+0000","last_fullsized":"2024-03-07T07:20:09.018003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:55:49.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018248+0000","last_change":"2024-03-07T07:20:08.028053+0000","last_active":"2024-03-07T07:20:09.018248+0000","last_peered":"2024-03-07T07:20:09.018248+0000","last_clean":"2024-03-07T07:20:09.018248+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:09.018248+0000","last_undegraded":"2024-03-07T07:20:09.018248+0000","last_fullsized":"2024-03-07T07:20:09.018248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.016495+0000","last_change":"2024-03-07T07:20:07.033411+0000","last_active":"2024-03-07T07:20:08.016495+0000","last_peered":"2024-03-07T07:20:08.016495+0000","last_clean":"2024-03-07T07:20:08.016495+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:08.016495+0000","last_undegraded":"2024-03-07T07:20:08.016495+0000","last_fullsized":"2024-03-07T07:20:08.016495+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:15.200144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040367+0000","last_change":"2024-03-07T07:20:08.032643+0000","last_active":"2024-03-07T07:20:08.040367+0000","last_peered":"2024-03-07T07:20:08.040367+0000","last_clean":"2024-03-07T07:20:08.040367+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:08.040367+0000","last_undegraded":"2024-03-07T07:20:08.040367+0000","last_fullsized":"2024-03-07T07:20:08.040367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018364+0000","last_change":"2024-03-07T07:20:07.027022+0000","last_active":"2024-03-07T07:20:09.018364+0000","last_peered":"2024-03-07T07:20:09.018364+0000","last_clean":"2024-03-07T07:20:09.018364+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:09.018364+0000","last_undegraded":"2024-03-07T07:20:09.018364+0000","last_fullsized":"2024-03-07T07:20:09.018364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:35.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032915+0000","last_change":"2024-03-07T07:20:08.028592+0000","last_active":"2024-03-07T07:20:08.032915+0000","last_peered":"2024-03-07T07:20:08.032915+0000","last_clean":"2024-03-07T07:20:08.032915+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:08.032915+0000","last_undegraded":"2024-03-07T07:20:08.032915+0000","last_fullsized":"2024-03-07T07:20:08.032915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.017522+0000","last_change":"2024-03-07T07:20:07.030367+0000","last_active":"2024-03-07T07:20:08.017522+0000","last_peered":"2024-03-07T07:20:08.017522+0000","last_clean":"2024-03-07T07:20:08.017522+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:08.017522+0000","last_undegraded":"2024-03-07T07:20:08.017522+0000","last_fullsized":"2024-03-07T07:20:08.017522+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:12.128114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041454+0000","last_change":"2024-03-07T07:20:08.034949+0000","last_active":"2024-03-07T07:20:08.041454+0000","last_peered":"2024-03-07T07:20:08.041454+0000","last_clean":"2024-03-07T07:20:08.041454+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:08.041454+0000","last_undegraded":"2024-03-07T07:20:08.041454+0000","last_fullsized":"2024-03-07T07:20:08.041454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015409+0000","last_change":"2024-03-07T07:20:07.030049+0000","last_active":"2024-03-07T07:20:09.015409+0000","last_peered":"2024-03-07T07:20:09.015409+0000","last_clean":"2024-03-07T07:20:09.015409+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:09.015409+0000","last_undegraded":"2024-03-07T07:20:09.015409+0000","last_fullsized":"2024-03-07T07:20:09.015409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.033701+0000","last_change":"2024-03-07T07:20:08.030884+0000","last_active":"2024-03-07T07:20:08.033701+0000","last_peered":"2024-03-07T07:20:08.033701+0000","last_clean":"2024-03-07T07:20:08.033701+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:08.033701+0000","last_undegraded":"2024-03-07T07:20:08.033701+0000","last_fullsized":"2024-03-07T07:20:08.033701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034520+0000","last_change":"2024-03-07T07:20:07.028526+0000","last_active":"2024-03-07T07:20:08.034520+0000","last_peered":"2024-03-07T07:20:08.034520+0000","last_clean":"2024-03-07T07:20:08.034520+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:08.034520+0000","last_undegraded":"2024-03-07T07:20:08.034520+0000","last_fullsized":"2024-03-07T07:20:08.034520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041419+0000","last_change":"2024-03-07T07:20:08.038594+0000","last_active":"2024-03-07T07:20:08.041419+0000","last_peered":"2024-03-07T07:20:08.041419+0000","last_clean":"2024-03-07T07:20:08.041419+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:08.041419+0000","last_undegraded":"2024-03-07T07:20:08.041419+0000","last_fullsized":"2024-03-07T07:20:08.041419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.048918+0000","last_change":"2024-03-07T07:20:07.035824+0000","last_active":"2024-03-07T07:20:08.048918+0000","last_peered":"2024-03-07T07:20:08.048918+0000","last_clean":"2024-03-07T07:20:08.048918+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:08.048918+0000","last_undegraded":"2024-03-07T07:20:08.048918+0000","last_fullsized":"2024-03-07T07:20:08.048918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043867+0000","last_change":"2024-03-07T07:20:08.034611+0000","last_active":"2024-03-07T07:20:08.043867+0000","last_peered":"2024-03-07T07:20:08.043867+0000","last_clean":"2024-03-07T07:20:08.043867+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:08.043867+0000","last_undegraded":"2024-03-07T07:20:08.043867+0000","last_fullsized":"2024-03-07T07:20:08.043867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018274+0000","last_change":"2024-03-07T07:20:07.025705+0000","last_active":"2024-03-07T07:20:09.018274+0000","last_peered":"2024-03-07T07:20:09.018274+0000","last_clean":"2024-03-07T07:20:09.018274+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:09.018274+0000","last_undegraded":"2024-03-07T07:20:09.018274+0000","last_fullsized":"2024-03-07T07:20:09.018274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042617+0000","last_change":"2024-03-07T07:20:08.037145+0000","last_active":"2024-03-07T07:20:08.042617+0000","last_peered":"2024-03-07T07:20:08.042617+0000","last_clean":"2024-03-07T07:20:08.042617+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:08.042617+0000","last_undegraded":"2024-03-07T07:20:08.042617+0000","last_fullsized":"2024-03-07T07:20:08.042617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.026553+0000","last_change":"2024-03-07T07:20:07.026780+0000","last_active":"2024-03-07T07:20:08.026553+0000","last_peered":"2024-03-07T07:20:08.026553+0000","last_clean":"2024-03-07T07:20:08.026553+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:08.026553+0000","last_undegraded":"2024-03-07T07:20:08.026553+0000","last_fullsized":"2024-03-07T07:20:08.026553+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039355+0000","last_change":"2024-03-07T07:20:08.033869+0000","last_active":"2024-03-07T07:20:08.039355+0000","last_peered":"2024-03-07T07:20:08.039355+0000","last_clean":"2024-03-07T07:20:08.039355+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:08.039355+0000","last_undegraded":"2024-03-07T07:20:08.039355+0000","last_fullsized":"2024-03-07T07:20:08.039355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.017152+0000","last_change":"2024-03-07T07:20:07.032636+0000","last_active":"2024-03-07T07:20:08.017152+0000","last_peered":"2024-03-07T07:20:08.017152+0000","last_clean":"2024-03-07T07:20:08.017152+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:08.017152+0000","last_undegraded":"2024-03-07T07:20:08.017152+0000","last_fullsized":"2024-03-07T07:20:08.017152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017781+0000","last_change":"2024-03-07T07:20:08.036782+0000","last_active":"2024-03-07T07:20:09.017781+0000","last_peered":"2024-03-07T07:20:09.017781+0000","last_clean":"2024-03-07T07:20:09.017781+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:09.017781+0000","last_undegraded":"2024-03-07T07:20:09.017781+0000","last_fullsized":"2024-03-07T07:20:09.017781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:19.197402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.042703+0000","last_change":"2024-03-07T07:20:07.033230+0000","last_active":"2024-03-07T07:20:08.042703+0000","last_peered":"2024-03-07T07:20:08.042703+0000","last_clean":"2024-03-07T07:20:08.042703+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:08.042703+0000","last_undegraded":"2024-03-07T07:20:08.042703+0000","last_fullsized":"2024-03-07T07:20:08.042703+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:42.777795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018625+0000","last_change":"2024-03-07T07:20:08.036934+0000","last_active":"2024-03-07T07:20:09.018625+0000","last_peered":"2024-03-07T07:20:09.018625+0000","last_clean":"2024-03-07T07:20:09.018625+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:09.018625+0000","last_undegraded":"2024-03-07T07:20:09.018625+0000","last_fullsized":"2024-03-07T07:20:09.018625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043377+0000","last_change":"2024-03-07T07:20:07.033018+0000","last_active":"2024-03-07T07:20:08.043377+0000","last_peered":"2024-03-07T07:20:08.043377+0000","last_clean":"2024-03-07T07:20:08.043377+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:08.043377+0000","last_undegraded":"2024-03-07T07:20:08.043377+0000","last_fullsized":"2024-03-07T07:20:08.043377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040666+0000","last_change":"2024-03-07T07:20:08.035720+0000","last_active":"2024-03-07T07:20:08.040666+0000","last_peered":"2024-03-07T07:20:08.040666+0000","last_clean":"2024-03-07T07:20:08.040666+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:08.040666+0000","last_undegraded":"2024-03-07T07:20:08.040666+0000","last_fullsized":"2024-03-07T07:20:08.040666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018650+0000","last_change":"2024-03-07T07:20:07.026560+0000","last_active":"2024-03-07T07:20:09.018650+0000","last_peered":"2024-03-07T07:20:09.018650+0000","last_clean":"2024-03-07T07:20:09.018650+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:09.018650+0000","last_undegraded":"2024-03-07T07:20:09.018650+0000","last_fullsized":"2024-03-07T07:20:09.018650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:52.614111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.043805+0000","last_change":"2024-03-07T07:20:08.033852+0000","last_active":"2024-03-07T07:20:08.043805+0000","last_peered":"2024-03-07T07:20:08.043805+0000","last_clean":"2024-03-07T07:20:08.043805+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:08.043805+0000","last_undegraded":"2024-03-07T07:20:08.043805+0000","last_fullsized":"2024-03-07T07:20:08.043805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018510+0000","last_change":"2024-03-07T07:20:07.023668+0000","last_active":"2024-03-07T07:20:09.018510+0000","last_peered":"2024-03-07T07:20:09.018510+0000","last_clean":"2024-03-07T07:20:09.018510+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:09.018510+0000","last_undegraded":"2024-03-07T07:20:09.018510+0000","last_fullsized":"2024-03-07T07:20:09.018510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:07.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.033122+0000","last_change":"2024-03-07T07:20:08.030524+0000","last_active":"2024-03-07T07:20:08.033122+0000","last_peered":"2024-03-07T07:20:08.033122+0000","last_clean":"2024-03-07T07:20:08.033122+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:08.033122+0000","last_undegraded":"2024-03-07T07:20:08.033122+0000","last_fullsized":"2024-03-07T07:20:08.033122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:17:40.279427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018550+0000","last_change":"2024-03-07T07:20:07.029892+0000","last_active":"2024-03-07T07:20:09.018550+0000","last_peered":"2024-03-07T07:20:09.018550+0000","last_clean":"2024-03-07T07:20:09.018550+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:09.018550+0000","last_undegraded":"2024-03-07T07:20:09.018550+0000","last_fullsized":"2024-03-07T07:20:09.018550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:24.735113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034183+0000","last_change":"2024-03-07T07:20:08.032134+0000","last_active":"2024-03-07T07:20:08.034183+0000","last_peered":"2024-03-07T07:20:08.034183+0000","last_clean":"2024-03-07T07:20:08.034183+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:08.034183+0000","last_undegraded":"2024-03-07T07:20:08.034183+0000","last_fullsized":"2024-03-07T07:20:08.034183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:08.669181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015375+0000","last_change":"2024-03-07T07:20:07.028153+0000","last_active":"2024-03-07T07:20:09.015375+0000","last_peered":"2024-03-07T07:20:09.015375+0000","last_clean":"2024-03-07T07:20:09.015375+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:09.015375+0000","last_undegraded":"2024-03-07T07:20:09.015375+0000","last_fullsized":"2024-03-07T07:20:09.015375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.034156+0000","last_change":"2024-03-07T07:20:08.032594+0000","last_active":"2024-03-07T07:20:08.034156+0000","last_peered":"2024-03-07T07:20:08.034156+0000","last_clean":"2024-03-07T07:20:08.034156+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:08.034156+0000","last_undegraded":"2024-03-07T07:20:08.034156+0000","last_fullsized":"2024-03-07T07:20:08.034156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:36.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018303+0000","last_change":"2024-03-07T07:20:07.028455+0000","last_active":"2024-03-07T07:20:09.018303+0000","last_peered":"2024-03-07T07:20:09.018303+0000","last_clean":"2024-03-07T07:20:09.018303+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:09.018303+0000","last_undegraded":"2024-03-07T07:20:09.018303+0000","last_fullsized":"2024-03-07T07:20:09.018303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.033039+0000","last_change":"2024-03-07T07:20:08.031299+0000","last_active":"2024-03-07T07:20:08.033039+0000","last_peered":"2024-03-07T07:20:08.033039+0000","last_clean":"2024-03-07T07:20:08.033039+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:08.033039+0000","last_undegraded":"2024-03-07T07:20:08.033039+0000","last_fullsized":"2024-03-07T07:20:08.033039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:37:06.166772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017852+0000","last_change":"2024-03-07T07:20:07.028369+0000","last_active":"2024-03-07T07:20:09.017852+0000","last_peered":"2024-03-07T07:20:09.017852+0000","last_clean":"2024-03-07T07:20:09.017852+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:09.017852+0000","last_undegraded":"2024-03-07T07:20:09.017852+0000","last_fullsized":"2024-03-07T07:20:09.017852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.038359+0000","last_change":"2024-03-07T07:20:08.033176+0000","last_active":"2024-03-07T07:20:08.038359+0000","last_peered":"2024-03-07T07:20:08.038359+0000","last_clean":"2024-03-07T07:20:08.038359+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:08.038359+0000","last_undegraded":"2024-03-07T07:20:08.038359+0000","last_fullsized":"2024-03-07T07:20:08.038359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.017914+0000","last_change":"2024-03-07T07:20:07.029244+0000","last_active":"2024-03-07T07:20:08.017914+0000","last_peered":"2024-03-07T07:20:08.017914+0000","last_clean":"2024-03-07T07:20:08.017914+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:08.017914+0000","last_undegraded":"2024-03-07T07:20:08.017914+0000","last_fullsized":"2024-03-07T07:20:08.017914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.032747+0000","last_change":"2024-03-07T07:20:08.026414+0000","last_active":"2024-03-07T07:20:08.032747+0000","last_peered":"2024-03-07T07:20:08.032747+0000","last_clean":"2024-03-07T07:20:08.032747+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:08.032747+0000","last_undegraded":"2024-03-07T07:20:08.032747+0000","last_fullsized":"2024-03-07T07:20:08.032747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:26:04.002320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041613+0000","last_change":"2024-03-07T07:20:07.027164+0000","last_active":"2024-03-07T07:20:08.041613+0000","last_peered":"2024-03-07T07:20:08.041613+0000","last_clean":"2024-03-07T07:20:08.041613+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:08.041613+0000","last_undegraded":"2024-03-07T07:20:08.041613+0000","last_fullsized":"2024-03-07T07:20:08.041613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.040644+0000","last_change":"2024-03-07T07:20:08.035488+0000","last_active":"2024-03-07T07:20:08.040644+0000","last_peered":"2024-03-07T07:20:08.040644+0000","last_clean":"2024-03-07T07:20:08.040644+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:08.040644+0000","last_undegraded":"2024-03-07T07:20:08.040644+0000","last_fullsized":"2024-03-07T07:20:08.040644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015456+0000","last_change":"2024-03-07T07:20:07.027534+0000","last_active":"2024-03-07T07:20:09.015456+0000","last_peered":"2024-03-07T07:20:09.015456+0000","last_clean":"2024-03-07T07:20:09.015456+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:09.015456+0000","last_undegraded":"2024-03-07T07:20:09.015456+0000","last_fullsized":"2024-03-07T07:20:09.015456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015303+0000","last_change":"2024-03-07T07:20:08.036269+0000","last_active":"2024-03-07T07:20:09.015303+0000","last_peered":"2024-03-07T07:20:09.015303+0000","last_clean":"2024-03-07T07:20:09.015303+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:09.015303+0000","last_undegraded":"2024-03-07T07:20:09.015303+0000","last_fullsized":"2024-03-07T07:20:09.015303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.025886+0000","last_change":"2024-03-07T07:20:07.023056+0000","last_active":"2024-03-07T07:20:08.025886+0000","last_peered":"2024-03-07T07:20:08.025886+0000","last_clean":"2024-03-07T07:20:08.025886+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:08.025886+0000","last_undegraded":"2024-03-07T07:20:08.025886+0000","last_fullsized":"2024-03-07T07:20:08.025886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018197+0000","last_change":"2024-03-07T07:20:08.023363+0000","last_active":"2024-03-07T07:20:09.018197+0000","last_peered":"2024-03-07T07:20:09.018197+0000","last_clean":"2024-03-07T07:20:09.018197+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:09.018197+0000","last_undegraded":"2024-03-07T07:20:09.018197+0000","last_fullsized":"2024-03-07T07:20:09.018197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015478+0000","last_change":"2024-03-07T07:20:07.024131+0000","last_active":"2024-03-07T07:20:09.015478+0000","last_peered":"2024-03-07T07:20:09.015478+0000","last_clean":"2024-03-07T07:20:09.015478+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:09.015478+0000","last_undegraded":"2024-03-07T07:20:09.015478+0000","last_fullsized":"2024-03-07T07:20:09.015478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015390+0000","last_change":"2024-03-07T07:20:08.030007+0000","last_active":"2024-03-07T07:20:09.015390+0000","last_peered":"2024-03-07T07:20:09.015390+0000","last_clean":"2024-03-07T07:20:09.015390+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:09.015390+0000","last_undegraded":"2024-03-07T07:20:09.015390+0000","last_fullsized":"2024-03-07T07:20:09.015390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.026687+0000","last_change":"2024-03-07T07:20:07.025544+0000","last_active":"2024-03-07T07:20:08.026687+0000","last_peered":"2024-03-07T07:20:08.026687+0000","last_clean":"2024-03-07T07:20:08.026687+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:08.026687+0000","last_undegraded":"2024-03-07T07:20:08.026687+0000","last_fullsized":"2024-03-07T07:20:08.026687+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:18.193822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.041157+0000","last_change":"2024-03-07T07:20:08.035274+0000","last_active":"2024-03-07T07:20:08.041157+0000","last_peered":"2024-03-07T07:20:08.041157+0000","last_clean":"2024-03-07T07:20:08.041157+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:08.041157+0000","last_undegraded":"2024-03-07T07:20:08.041157+0000","last_fullsized":"2024-03-07T07:20:08.041157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017946+0000","last_change":"2024-03-07T07:20:07.023385+0000","last_active":"2024-03-07T07:20:09.017946+0000","last_peered":"2024-03-07T07:20:09.017946+0000","last_clean":"2024-03-07T07:20:09.017946+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:09.017946+0000","last_undegraded":"2024-03-07T07:20:09.017946+0000","last_fullsized":"2024-03-07T07:20:09.017946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017838+0000","last_change":"2024-03-07T07:20:08.040036+0000","last_active":"2024-03-07T07:20:09.017838+0000","last_peered":"2024-03-07T07:20:09.017838+0000","last_clean":"2024-03-07T07:20:09.017838+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:09.017838+0000","last_undegraded":"2024-03-07T07:20:09.017838+0000","last_fullsized":"2024-03-07T07:20:09.017838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.016780+0000","last_change":"2024-03-07T07:20:07.036824+0000","last_active":"2024-03-07T07:20:08.016780+0000","last_peered":"2024-03-07T07:20:08.016780+0000","last_clean":"2024-03-07T07:20:08.016780+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:08.016780+0000","last_undegraded":"2024-03-07T07:20:08.016780+0000","last_fullsized":"2024-03-07T07:20:08.016780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:20.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.039292+0000","last_change":"2024-03-07T07:20:08.034942+0000","last_active":"2024-03-07T07:20:08.039292+0000","last_peered":"2024-03-07T07:20:08.039292+0000","last_clean":"2024-03-07T07:20:08.039292+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:08.039292+0000","last_undegraded":"2024-03-07T07:20:08.039292+0000","last_fullsized":"2024-03-07T07:20:08.039292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017654+0000","last_change":"2024-03-07T07:20:07.023606+0000","last_active":"2024-03-07T07:20:09.017654+0000","last_peered":"2024-03-07T07:20:09.017654+0000","last_clean":"2024-03-07T07:20:09.017654+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:09.017654+0000","last_undegraded":"2024-03-07T07:20:09.017654+0000","last_fullsized":"2024-03-07T07:20:09.017654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-07T07:20:08.026731+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:20:08.026731+0000","last_peered":"2024-03-07T07:20:08.026731+0000","last_clean":"2024-03-07T07:20:08.026731+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:08.026731+0000","last_undegraded":"2024-03-07T07:20:08.026731+0000","last_fullsized":"2024-03-07T07:20:08.026731+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":730774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":730684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:20:11.400 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-07T07:20:11.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-07T07:20:11.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.576+0000 7f973ab1d640 1 Processor -- start 2024-03-07T07:20:11.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.576+0000 7f973ab1d640 1 -- start start 2024-03-07T07:20:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f973ab1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f973415c330 0x7f97340acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f973ab1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97340b4670 0x7f97340b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f973ab1d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97341601b0 con 0x7f973415fd30 2024-03-07T07:20:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f973ab1d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9734160480 con 0x7f973415c330 2024-03-07T07:20:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f973ab1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9734160760 con 0x7f97340b4670 2024-03-07T07:20:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97388b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f973415c330 0x7f97340acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97388b9640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f973415c330 0x7f97340acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60320/0 (socket says 172.21.15.64:60320) 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97388b9640 1 -- 172.21.15.64:0/2586316803 learned_addr learned my addr 172.21.15.64:0/2586316803 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97390ba640 1 --2- 172.21.15.64:0/2586316803 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97340b4670 0x7f97340b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97388b9640 1 -- 172.21.15.64:0/2586316803 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97340b4670 msgr2=0x7f97340b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97388b9640 1 --2- 172.21.15.64:0/2586316803 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97340b4670 0x7f97340b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97388b9640 1 -- 172.21.15.64:0/2586316803 >> v1:172.21.15.120:6789/0 conn(0x7f973415fd30 legacy=0x7f97340cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97388b9640 1 -- 172.21.15.64:0/2586316803 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734160a40 con 0x7f973415c330 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.577+0000 7f97390ba640 1 --2- 172.21.15.64:0/2586316803 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97340b4670 0x7f97340b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:11.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.578+0000 7f97388b9640 1 --2- 172.21.15.64:0/2586316803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415c330 0x7f97340acd90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f972c009870 tx=0x7f972c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4281e049af51bc90 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f97337fe640 1 -- 172.21.15.64:0/2586316803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c035020 con 0x7f973415c330 2024-03-07T07:20:11.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f97337fe640 1 -- 172.21.15.64:0/2586316803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c031640 con 0x7f973415c330 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f97337fe640 1 -- 172.21.15.64:0/2586316803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c02d040 con 0x7f973415c330 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 -- 172.21.15.64:0/2586316803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415c330 msgr2=0x7f97340acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 --2- 172.21.15.64:0/2586316803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415c330 0x7f97340acd90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f972c009870 tx=0x7f972c02b8f0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 -- 172.21.15.64:0/2586316803 shutdown_connections 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 --2- 172.21.15.64:0/2586316803 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97340b4670 0x7f97340b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 --2- 172.21.15.64:0/2586316803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415c330 0x7f97340acd90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 -- 172.21.15.64:0/2586316803 >> 172.21.15.64:0/2586316803 conn(0x7f9734070400 msgr2=0x7f97340707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 -- 172.21.15.64:0/2586316803 shutdown_connections 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.579+0000 7f973ab1d640 1 -- 172.21.15.64:0/2586316803 wait complete. 2024-03-07T07:20:11.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.580+0000 7f973ab1d640 1 Processor -- start 2024-03-07T07:20:11.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.580+0000 7f973ab1d640 1 -- start start 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f973ab1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97340b4670 0x7f9734151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f973ab1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f973415c330 0x7f9734072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f973ab1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415fd30 0x7f9734072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f973ab1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97340b33c0 con 0x7f973415fd30 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f973ab1d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97340cd160 con 0x7f97340b4670 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f973ab1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97340b3540 con 0x7f973415c330 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f97390ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415fd30 0x7f9734072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f97390ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415fd30 0x7f9734072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60336/0 (socket says 172.21.15.64:60336) 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.581+0000 7f97390ba640 1 -- 172.21.15.64:0/1397515298 learned_addr learned my addr 172.21.15.64:0/1397515298 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f9733fff640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f973415c330 0x7f9734072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97388b9640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97340b4670 0x7f9734151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97390ba640 1 -- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f973415c330 msgr2=0x7f9734072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97390ba640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f973415c330 0x7f9734072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97390ba640 1 -- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97340b4670 msgr2=0x7f9734151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97390ba640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97340b4670 0x7f9734151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97390ba640 1 -- 172.21.15.64:0/1397515298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97341f4de0 con 0x7f973415fd30 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97390ba640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415fd30 0x7f9734072b60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f972400cef0 tx=0x7f972400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97317fa640 1 -- 172.21.15.64:0/1397515298 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972401a070 con 0x7f973415fd30 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97317fa640 1 -- 172.21.15.64:0/1397515298 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972400d860 con 0x7f973415fd30 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f97317fa640 1 -- 172.21.15.64:0/1397515298 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972400dc50 con 0x7f973415fd30 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97341f50c0 con 0x7f973415fd30 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.582+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97341f5960 con 0x7f973415fd30 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.583+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97341f5b60 con 0x7f973415fd30 2024-03-07T07:20:11.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.584+0000 7f97317fa640 1 -- 172.21.15.64:0/1397515298 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f972401d070 con 0x7f973415fd30 2024-03-07T07:20:11.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.584+0000 7f97317fa640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f972006b500 0x7f972006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:11.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.584+0000 7f97317fa640 1 -- 172.21.15.64:0/1397515298 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f972408a9d0 con 0x7f973415fd30 2024-03-07T07:20:11.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.584+0000 7f97388b9640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f972006b500 0x7f972006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:11.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.585+0000 7f97388b9640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f972006b500 0x7f972006d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f972c036000 tx=0x7f972c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:11.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.593+0000 7f97317fa640 1 -- 172.21.15.64:0/1397515298 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f97341f5b60 con 0x7f973415fd30 2024-03-07T07:20:11.709 INFO:tasks.ceph.mds.c.smithi064.stderr:2024-03-07T07:20:11.708+0000 7f65685f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T07:20:11.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.921+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9734062c90 con 0x7f973415fd30 2024-03-07T07:20:11.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.923+0000 7f97317fa640 1 -- 172.21.15.64:0/1397515298 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f97240592c0 con 0x7f973415fd30 2024-03-07T07:20:11.924 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-03-07T07:20:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQA6aullg6DUIhAAM9uoMC2xuAIACpfgwkhF9A== 2024-03-07T07:20:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: caps mds = "allow" 2024-03-07T07:20:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: caps mon = "allow r" 2024-03-07T07:20:11.924 INFO:teuthology.orchestra.run.smithi064.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-07T07:20:11.924 INFO:teuthology.orchestra.run.smithi064.stderr:updated caps for client.0 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f972006b500 msgr2=0x7f972006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f972006b500 0x7f972006d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f972c036000 tx=0x7f972c03c040 comp rx=0 tx=0).stop 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415fd30 msgr2=0x7f9734072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415fd30 0x7f9734072b60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f972400cef0 tx=0x7f972400a710 comp rx=0 tx=0).stop 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 shutdown_connections 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f972006b500 0x7f972006d9c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f973415fd30 0x7f9734072b60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f973415c330 0x7f9734072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 --2- 172.21.15.64:0/1397515298 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97340b4670 0x7f9734151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 >> 172.21.15.64:0/1397515298 conn(0x7f9734070400 msgr2=0x7f97340ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 shutdown_connections 2024-03-07T07:20:11.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:11.925+0000 7f973ab1d640 1 -- 172.21.15.64:0/1397515298 wait complete. 2024-03-07T07:20:11.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:20:12.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.037+0000 7f17d13d6640 1 Processor -- start 2024-03-07T07:20:12.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.037+0000 7f17d13d6640 1 -- start start 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17d13d6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc0d1860 0x7f17cc06b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17d13d6640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f17cc0b7390 0x7f17cc06ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17d13d6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc0d0aa0 con 0x7f17cc145bf0 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17d13d6640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0d13d0 con 0x7f17cc0b7390 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17d13d6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0d0d80 con 0x7f17cc0d1860 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17cb7fe640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f17cc0b7390 0x7f17cc06ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17caffd640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f17cc145bf0 0x7f17cc0cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60154/0 (socket says 172.21.15.64:60154) 2024-03-07T07:20:12.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17caffd640 1 -- 172.21.15.64:0/1090739740 learned_addr learned my addr 172.21.15.64:0/1090739740 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17ca7fc640 1 --2- 172.21.15.64:0/1090739740 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc0d1860 0x7f17cc06b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17cb7fe640 1 -- 172.21.15.64:0/1090739740 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc0d1860 msgr2=0x7f17cc06b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17cb7fe640 1 --2- 172.21.15.64:0/1090739740 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc0d1860 0x7f17cc06b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17cb7fe640 1 -- 172.21.15.64:0/1090739740 >> v1:172.21.15.64:6789/0 conn(0x7f17cc145bf0 legacy=0x7f17cc0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17cb7fe640 1 -- 172.21.15.64:0/1090739740 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0d1060 con 0x7f17cc0b7390 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17c9ffb640 1 -- 172.21.15.64:0/1090739740 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001351810 0 0) 0x7f17cc0d0aa0 con 0x7f17cc145bf0 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.038+0000 7f17ca7fc640 1 --2- 172.21.15.64:0/1090739740 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc0d1860 0x7f17cc06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.039+0000 7f17cb7fe640 1 --2- 172.21.15.64:0/1090739740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0b7390 0x7f17cc06ba50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0d1550 tx=0x7f17c402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0082aa71066581 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.039+0000 7f17c9ffb640 1 -- 172.21.15.64:0/1090739740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4039070 con 0x7f17cc0b7390 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.039+0000 7f17c9ffb640 1 -- 172.21.15.64:0/1090739740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c402cc40 con 0x7f17cc0b7390 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.039+0000 7f17c9ffb640 1 -- 172.21.15.64:0/1090739740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4034440 con 0x7f17cc0b7390 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 -- 172.21.15.64:0/1090739740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0b7390 msgr2=0x7f17cc06ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1090739740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0b7390 0x7f17cc06ba50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0d1550 tx=0x7f17c402be90 comp rx=0 tx=0).stop 2024-03-07T07:20:12.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 -- 172.21.15.64:0/1090739740 shutdown_connections 2024-03-07T07:20:12.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1090739740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0b7390 0x7f17cc06ba50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1090739740 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc0d1860 0x7f17cc06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 -- 172.21.15.64:0/1090739740 >> 172.21.15.64:0/1090739740 conn(0x7f17cc06d4c0 msgr2=0x7f17cc066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 -- 172.21.15.64:0/1090739740 shutdown_connections 2024-03-07T07:20:12.043 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.040+0000 7f17d13d6640 1 -- 172.21.15.64:0/1090739740 wait complete. 2024-03-07T07:20:12.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.041+0000 7f17d13d6640 1 Processor -- start 2024-03-07T07:20:12.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.041+0000 7f17d13d6640 1 -- start start 2024-03-07T07:20:12.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17d13d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17cc0b7390 0x7f17cc140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17d13d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0d1860 0x7f17cc140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17d13d6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc145bf0 0x7f17cc1f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17d13d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0d0840 con 0x7f17cc0d1860 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17d13d6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0d06c0 con 0x7f17cc0b7390 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17d13d6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0b5160 con 0x7f17cc145bf0 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0d1860 0x7f17cc140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0d1860 0x7f17cc140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60346/0 (socket says 172.21.15.64:60346) 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 -- 172.21.15.64:0/1875137349 learned_addr learned my addr 172.21.15.64:0/1875137349 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17caffd640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17cc0b7390 0x7f17cc140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17cb7fe640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc145bf0 0x7f17cc1f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 -- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc145bf0 msgr2=0x7f17cc1f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc145bf0 0x7f17cc1f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 -- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17cc0b7390 msgr2=0x7f17cc140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17cc0b7390 0x7f17cc140500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17ca7fc640 1 -- 172.21.15.64:0/1875137349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc15ca20 con 0x7f17cc0d1860 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.042+0000 7f17caffd640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17cc0b7390 0x7f17cc140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:12.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.043+0000 7f17ca7fc640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0d1860 0x7f17cc140a40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c7f0 tx=0x7f17c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.043+0000 7f17b37fe640 1 -- 172.21.15.64:0/1875137349 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0015070 con 0x7f17cc0d1860 2024-03-07T07:20:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.043+0000 7f17b37fe640 1 -- 172.21.15.64:0/1875137349 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0007a30 con 0x7f17cc0d1860 2024-03-07T07:20:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.043+0000 7f17b37fe640 1 -- 172.21.15.64:0/1875137349 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0007e00 con 0x7f17cc0d1860 2024-03-07T07:20:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.043+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc15cd00 con 0x7f17cc0d1860 2024-03-07T07:20:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.043+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc15d2c0 con 0x7f17cc0d1860 2024-03-07T07:20:12.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.044+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc062c90 con 0x7f17cc0d1860 2024-03-07T07:20:12.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.045+0000 7f17b37fe640 1 -- 172.21.15.64:0/1875137349 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f17c0018070 con 0x7f17cc0d1860 2024-03-07T07:20:12.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.045+0000 7f17b37fe640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17b806b4d0 0x7f17b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.045+0000 7f17caffd640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17b806b4d0 0x7f17b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.046+0000 7f17b37fe640 1 -- 172.21.15.64:0/1875137349 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17c0088710 con 0x7f17cc0d1860 2024-03-07T07:20:12.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.046+0000 7f17caffd640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17b806b4d0 0x7f17b806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000f60 tx=0x7f17bc007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.055+0000 7f17b37fe640 1 -- 172.21.15.64:0/1875137349 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f17c0058900 con 0x7f17cc0d1860 2024-03-07T07:20:12.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.273+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17cc0ceae0 con 0x7f17cc0d1860 2024-03-07T07:20:12.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.273+0000 7f17b37fe640 1 -- 172.21.15.64:0/1875137349 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f17c005c9a0 con 0x7f17cc0d1860 2024-03-07T07:20:12.276 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:12.277 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/1229777071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1229777071},{"type":"v1","addr":"172.21.15.120:6836","nonce":1229777071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3785220969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":3785220969},{"type":"v1","addr":"172.21.15.120:6837","nonce":3785220969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/229489259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":229489259},{"type":"v1","addr":"172.21.15.64:6834","nonce":229489259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:20:07.375701+0000","modified":"2024-03-07T07:20:09.484751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3131353555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3131353555},{"type":"v1","addr":"172.21.15.64:6835","nonce":3131353555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-03-07T07:20:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-03-07T07:20:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17b806b4d0 msgr2=0x7f17b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17b806b4d0 0x7f17b806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000f60 tx=0x7f17bc007ca0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0d1860 msgr2=0x7f17cc140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0d1860 0x7f17cc140a40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c7f0 tx=0x7f17c000ccb0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 shutdown_connections 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f17b806b4d0 0x7f17b806d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17cc0d1860 0x7f17cc140a40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17cc145bf0 0x7f17cc1f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 --2- 172.21.15.64:0/1875137349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17cc0b7390 0x7f17cc140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 >> 172.21.15.64:0/1875137349 conn(0x7f17cc06d4c0 msgr2=0x7f17cc141830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 shutdown_connections 2024-03-07T07:20:12.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.275+0000 7f17d13d6640 1 -- 172.21.15.64:0/1875137349 wait complete. 2024-03-07T07:20:12.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-07T07:20:07.375701+0000', 'modified': '2024-03-07T07:20:09.484751+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5082}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5082': {'gid': 5082, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6835/3131353555', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6833', 'nonce': 3131353555}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 3131353555}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5082, 'qdb_cluster': [5082]} 2024-03-07T07:20:12.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-07T07:20:12.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-07T07:20:12.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.792+0000 7f46db3a7640 1 Processor -- start 2024-03-07T07:20:12.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.793+0000 7f46db3a7640 1 -- start start 2024-03-07T07:20:12.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.793+0000 7f46db3a7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f46d4147270 0x7f46d406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.793+0000 7f46db3a7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f46d4147810 0x7f46d406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.793+0000 7f46db3a7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d40d1e70 con 0x7f46d40d2c10 2024-03-07T07:20:12.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.793+0000 7f46db3a7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d4147650 con 0x7f46d4147270 2024-03-07T07:20:12.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.793+0000 7f46db3a7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d40b7520 con 0x7f46d4147810 2024-03-07T07:20:12.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d9143640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f46d4147270 0x7f46d406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d8942640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f46d40d2c10 0x7f46d4162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60170/0 (socket says 172.21.15.64:60170) 2024-03-07T07:20:12.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d8942640 1 -- 172.21.15.64:0/2535119388 learned_addr learned my addr 172.21.15.64:0/2535119388 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:12.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d9944640 1 --2- 172.21.15.64:0/2535119388 >> v2:172.21.15.120:3300/0 conn(0x7f46d4147810 0x7f46d406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d9143640 1 -- 172.21.15.64:0/2535119388 >> v2:172.21.15.120:3300/0 conn(0x7f46d4147810 msgr2=0x7f46d406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d9143640 1 --2- 172.21.15.64:0/2535119388 >> v2:172.21.15.120:3300/0 conn(0x7f46d4147810 0x7f46d406ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d9143640 1 -- 172.21.15.64:0/2535119388 >> v1:172.21.15.64:6789/0 conn(0x7f46d40d2c10 legacy=0x7f46d4162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d3fff640 1 -- 172.21.15.64:0/2535119388 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171820723 0 0) 0x7f46d40d1e70 con 0x7f46d40d2c10 2024-03-07T07:20:12.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.794+0000 7f46d9143640 1 -- 172.21.15.64:0/2535119388 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40d2130 con 0x7f46d4147270 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46d9143640 1 --2- 172.21.15.64:0/2535119388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d406b510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f46d4073700 tx=0x7f46cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f620a6d3dcfcaf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46d3fff640 1 -- 172.21.15.64:0/2535119388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc035020 con 0x7f46d4147270 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46d3fff640 1 -- 172.21.15.64:0/2535119388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc031830 con 0x7f46d4147270 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46d3fff640 1 -- 172.21.15.64:0/2535119388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc031c00 con 0x7f46d4147270 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46db3a7640 1 -- 172.21.15.64:0/2535119388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 msgr2=0x7f46d406b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46db3a7640 1 --2- 172.21.15.64:0/2535119388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d406b510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f46d4073700 tx=0x7f46cc02b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46db3a7640 1 -- 172.21.15.64:0/2535119388 shutdown_connections 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46db3a7640 1 --2- 172.21.15.64:0/2535119388 >> v2:172.21.15.120:3300/0 conn(0x7f46d4147810 0x7f46d406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46db3a7640 1 --2- 172.21.15.64:0/2535119388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d406b510 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.795+0000 7f46db3a7640 1 -- 172.21.15.64:0/2535119388 >> 172.21.15.64:0/2535119388 conn(0x7f46d406d4c0 msgr2=0x7f46d4066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.796+0000 7f46db3a7640 1 -- 172.21.15.64:0/2535119388 shutdown_connections 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.796+0000 7f46db3a7640 1 -- 172.21.15.64:0/2535119388 wait complete. 2024-03-07T07:20:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 Processor -- start 2024-03-07T07:20:12.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 -- start start 2024-03-07T07:20:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d40d2c10 0x7f46d413b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d413bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4147810 0x7f46d4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40b71d0 con 0x7f46d4147270 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40d26f0 con 0x7f46d40d2c10 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46db3a7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4147d60 con 0x7f46d4147810 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46d8942640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d413bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.797+0000 7f46d8942640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d413bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60362/0 (socket says 172.21.15.64:60362) 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d8942640 1 -- 172.21.15.64:0/3138342929 learned_addr learned my addr 172.21.15.64:0/3138342929 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d9143640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d40d2c10 0x7f46d413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d9944640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4147810 0x7f46d4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d8942640 1 -- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4147810 msgr2=0x7f46d4156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d8942640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4147810 0x7f46d4156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d8942640 1 -- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d40d2c10 msgr2=0x7f46d413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d8942640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d40d2c10 0x7f46d413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d8942640 1 -- 172.21.15.64:0/3138342929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d4157320 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d9143640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d40d2c10 0x7f46d413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d9944640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4147810 0x7f46d4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d8942640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d413bdb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c4d0 tx=0x7f46c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d1ffb640 1 -- 172.21.15.64:0/3138342929 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8018070 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d1ffb640 1 -- 172.21.15.64:0/3138342929 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c8007500 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46d1ffb640 1 -- 172.21.15.64:0/3138342929 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004a20 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d413c520 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.798+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d413cae0 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.800+0000 7f46d1ffb640 1 -- 172.21.15.64:0/3138342929 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f46c801b070 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.800+0000 7f46d1ffb640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46c006b4d0 0x7f46c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.800+0000 7f46d1ffb640 1 -- 172.21.15.64:0/3138342929 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46c8089490 con 0x7f46d4147270 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.800+0000 7f46d9143640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46c006b4d0 0x7f46c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.801+0000 7f46d9143640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46c006b4d0 0x7f46c006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46cc036000 tx=0x7f46cc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.801+0000 7f46de18a740 1 Processor -- start 2024-03-07T07:20:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.801+0000 7f46de18a740 1 -- start start 2024-03-07T07:20:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763655b0 0x564b7638c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b7638c620 0x564b763645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46de18a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76364b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46de18a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b762c9790 con 0x564b76365990 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b76272b20 con 0x564b763655b0 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b762c5830 con 0x564b7638c620 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76364b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76364b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60366/0 (socket says 172.21.15.64:60366) 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 -- 172.21.15.64:0/4101555479 learned_addr learned my addr 172.21.15.64:0/4101555479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9143640 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763655b0 0x564b7638c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d8942640 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b7638c620 0x564b763645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 -- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b7638c620 msgr2=0x564b763645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b7638c620 0x564b763645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 -- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763655b0 msgr2=0x564b7638c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763655b0 0x564b7638c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 -- 172.21.15.64:0/4101555479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b76286970 con 0x564b76365990 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.802+0000 7f46d9944640 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76364b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f46c4007330 tx=0x7f46c40077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9cbed538f780d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46baffd640 1 -- 172.21.15.64:0/4101555479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c401a070 con 0x564b76365990 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46baffd640 1 -- 172.21.15.64:0/4101555479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c4003da0 con 0x564b76365990 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46baffd640 1 -- 172.21.15.64:0/4101555479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4004230 con 0x564b76365990 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 -- 172.21.15.64:0/4101555479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 msgr2=0x564b76364b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76364b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f46c4007330 tx=0x7f46c40077f0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 -- 172.21.15.64:0/4101555479 shutdown_connections 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76364b30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b7638c620 0x564b763645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 --2- 172.21.15.64:0/4101555479 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763655b0 0x564b7638c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 -- 172.21.15.64:0/4101555479 >> 172.21.15.64:0/4101555479 conn(0x564b7632c3d0 msgr2=0x564b763aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 -- 172.21.15.64:0/4101555479 shutdown_connections 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.803+0000 7f46de18a740 1 -- 172.21.15.64:0/4101555479 wait complete. 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.804+0000 7f46de18a740 1 -- start start 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.804+0000 7f46de18a740 1 Processor -- start 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46de18a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b763655b0 0x564b76389d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b76365990 0x564b763ba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763836a0 0x564b763ba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46de18a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7638bc50 con 0x564b763655b0 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b76285f60 con 0x564b76365990 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b763849d0 con 0x564b763836a0 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46de18a740 10 client.0 _ll_register_callbacks cb 0x564b762c13d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b763655b0 0x564b76389d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b763655b0 0x564b76389d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60370/0 (socket says 172.21.15.64:60370) 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 -- 172.21.15.64:0/4174241949 learned_addr learned my addr 172.21.15.64:0/4174241949 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d8942640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b76365990 0x564b763ba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9944640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763836a0 0x564b763ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 -- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763836a0 msgr2=0x564b763ba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763836a0 0x564b763ba570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 -- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b76365990 msgr2=0x564b763ba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b76365990 0x564b763ba030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 -- 172.21.15.64:0/4174241949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b75fde2d0 con 0x564b763655b0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9944640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763836a0 0x564b763ba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46d9143640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b763655b0 0x564b76389d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009890 tx=0x7f46cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0ce8366c8f60a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.805+0000 7f46affff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46affff640 1 -- 172.21.15.64:0/4174241949 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc035040 con 0x564b763655b0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46affff640 1 -- 172.21.15.64:0/4174241949 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46cc003a20 con 0x564b763655b0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46c4003da0 con 0x564b763655b0 2024-03-07T07:20:12.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46affff640 1 -- 172.21.15.64:0/4174241949 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46cc003c20 con 0x564b763655b0 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 10 client.5265 fetch_fsmap learned FSMap version 14 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x564b760a86b0 con 0x564b763655b0 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46affff640 1 -- 172.21.15.64:0/4174241949 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c4003da0 con 0x564b763655b0 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46affff640 1 -- 172.21.15.64:0/4174241949 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 14) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f46cc05c080 con 0x564b763655b0 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 10 client.5265 fetch_fsmap finished waiting for FSMap version 14 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 10 client.5265 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 20 client.5265 populate_metadata read hostname 'smithi064' 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763876e0 0x564b763bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 4 client.5265 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 --> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b762d6cd0 con 0x564b763876e0 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.806+0000 7f46d8942640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763876e0 0x564b763bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.807+0000 7f46d8942640 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763876e0 0x564b763bdb10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e1bbf86272f2fa server_cookie=b0ae1593275f37e9 in_seq=0 out_seq=0 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.807+0000 7f46affff640 10 client.5265 ms_handle_connect on v2:172.21.15.64:6833/3131353555 2024-03-07T07:20:12.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.807+0000 7f46affff640 1 -- 172.21.15.64:0/4174241949 <== mds.0 v2:172.21.15.64:6833/3131353555 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564b762d6cd0 con 0x564b763876e0 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.807+0000 7f46affff640 10 client.5265 handle_command_reply: tid=0 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.807+0000 7f46de18a740 1 client.5265 shutdown 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b763655b0 msgr2=0x564b76389d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b763655b0 0x564b76389d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009890 tx=0x7f46cc004730 comp rx=0 tx=0).stop 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 shutdown_connections 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763876e0 0x564b763bdb10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763836a0 0x564b763ba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b76365990 0x564b763ba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 --2- 172.21.15.64:0/4174241949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b763655b0 0x564b76389d80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 >> 172.21.15.64:0/4174241949 conn(0x564b7632c3d0 msgr2=0x564b7638b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46affff640 0 client.5265 ms_handle_reset on v2:172.21.15.64:6833/3131353555 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 shutdown_connections 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 1 -- 172.21.15.64:0/4174241949 wait complete. 2024-03-07T07:20:12.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.808+0000 7f46de18a740 20 client.5265 trim_cache size 0 max 16384 2024-03-07T07:20:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 Processor -- start 2024-03-07T07:20:12.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 -- start start 2024-03-07T07:20:12.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 0x564b76386980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76387b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76388050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d41ff490 con 0x564b76365990 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d41f88b0 con 0x564b763836a0 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d41ff0e0 con 0x564b763655b0 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46d8942640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76387b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46d8942640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76387b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60384/0 (socket says 172.21.15.64:60384) 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.825+0000 7f46d8942640 1 -- 172.21.15.64:0/4154185079 learned_addr learned my addr 172.21.15.64:0/4154185079 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46d8942640 1 -- 172.21.15.64:0/4154185079 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 msgr2=0x564b76386980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46d8942640 1 --2- 172.21.15.64:0/4154185079 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 0x564b76386980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46d8942640 1 -- 172.21.15.64:0/4154185079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 msgr2=0x564b76388050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46d8942640 1 --2- 172.21.15.64:0/4154185079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76388050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46d8942640 1 -- 172.21.15.64:0/4154185079 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b7635b460 con 0x564b76365990 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46d8942640 1 --2- 172.21.15.64:0/4154185079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76387b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46c8087a70 tx=0x7f46c805a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e399cadb7d5bb8c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46ba7fc640 1 -- 172.21.15.64:0/4154185079 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8055fa0 con 0x564b76365990 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46ba7fc640 1 -- 172.21.15.64:0/4154185079 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c806f070 con 0x564b76365990 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46ba7fc640 1 -- 172.21.15.64:0/4154185079 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c805ff20 con 0x564b76365990 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46de18a740 1 -- 172.21.15.64:0/4154185079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 msgr2=0x564b76387b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.826+0000 7f46de18a740 1 --2- 172.21.15.64:0/4154185079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76387b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46c8087a70 tx=0x7f46c805a040 comp rx=0 tx=0).stop 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 -- 172.21.15.64:0/4154185079 shutdown_connections 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 --2- 172.21.15.64:0/4154185079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76387b10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 --2- 172.21.15.64:0/4154185079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76388050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 --2- 172.21.15.64:0/4154185079 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 0x564b76386980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 -- 172.21.15.64:0/4154185079 >> 172.21.15.64:0/4154185079 conn(0x564b7632c3d0 msgr2=0x564b76393140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 -- 172.21.15.64:0/4154185079 shutdown_connections 2024-03-07T07:20:12.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 -- 172.21.15.64:0/4154185079 wait complete. 2024-03-07T07:20:12.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 -- start start 2024-03-07T07:20:12.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.827+0000 7f46de18a740 1 Processor -- start 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 0x564b76388480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46de18a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76413950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46de18a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76413e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46de18a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7638bc50 con 0x564b76365990 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b763baab0 con 0x564b763836a0 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46de18a740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b76384d10 con 0x564b763655b0 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46de18a740 10 client.0 _ll_register_callbacks cb 0x7f46cc0041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76413950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76413950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60392/0 (socket says 172.21.15.64:60392) 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 -- 172.21.15.64:0/2117224328 learned_addr learned my addr 172.21.15.64:0/2117224328 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d9143640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 0x564b76388480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d9944640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76413e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 -- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 msgr2=0x564b76388480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 0x564b76388480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 -- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 msgr2=0x564b76413e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76413e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d8942640 1 -- 172.21.15.64:0/2117224328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b763889c0 con 0x564b76365990 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.828+0000 7f46d9944640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76413e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46d8942640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76413950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46c8054100 tx=0x7f46c8089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd12f64d7b883dbf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46affff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46affff640 1 -- 172.21.15.64:0/2117224328 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8055fa0 con 0x564b76365990 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46affff640 1 -- 172.21.15.64:0/2117224328 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46c8004bc0 con 0x564b76365990 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46c806f070 con 0x564b76365990 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46affff640 1 -- 172.21.15.64:0/2117224328 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c8004de0 con 0x564b76365990 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 10 client.5271 fetch_fsmap learned FSMap version 14 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x564b7635adc0 con 0x564b76365990 2024-03-07T07:20:12.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46affff640 1 -- 172.21.15.64:0/2117224328 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c806f070 con 0x564b76365990 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46affff640 1 -- 172.21.15.64:0/2117224328 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 14) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f46c8076b70 con 0x564b76365990 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 10 client.5271 fetch_fsmap finished waiting for FSMap version 14 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 10 client.5271 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 20 client.5271 populate_metadata read hostname 'smithi064' 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763a4520 0x564b763a4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 4 client.5271 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 --> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b7638c140 con 0x564b763a4520 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.829+0000 7f46d9143640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763a4520 0x564b763a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46d9143640 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763a4520 0x564b763a4900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900b37f6e0e3f1b2 server_cookie=4e82d8a726b16c67 in_seq=0 out_seq=0 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46affff640 10 client.5271 ms_handle_connect on v2:172.21.15.64:6833/3131353555 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46affff640 1 -- 172.21.15.64:0/2117224328 <== mds.0 v2:172.21.15.64:6833/3131353555 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x564b7638c140 con 0x564b763a4520 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46affff640 10 client.5271 handle_command_reply: tid=0 2024-03-07T07:20:12.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 client.5271 shutdown 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 msgr2=0x564b76413950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76413950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46c8054100 tx=0x7f46c8089040 comp rx=0 tx=0).stop 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 shutdown_connections 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:6833/3131353555,v1:172.21.15.64:6835/3131353555] conn(0x564b763a4520 0x564b763a4900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b76365990 0x564b76413950 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46affff640 0 client.5271 ms_handle_reset on v2:172.21.15.64:6833/3131353555 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b763836a0 0x564b76413e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 --2- 172.21.15.64:0/2117224328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x564b763655b0 0x564b76388480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 >> 172.21.15.64:0/2117224328 conn(0x564b7632c3d0 msgr2=0x564b76386980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 shutdown_connections 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 1 -- 172.21.15.64:0/2117224328 wait complete. 2024-03-07T07:20:12.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.830+0000 7f46de18a740 20 client.5271 trim_cache size 0 max 16384 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3", 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 5082, 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 4.819297905, 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 14, 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 33, 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 33, 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 11.979447541000001 2024-03-07T07:20:12.838 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:12.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46c006b4d0 msgr2=0x7f46c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46c006b4d0 0x7f46c006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46cc036000 tx=0x7f46cc008030 comp rx=0 tx=0).stop 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 msgr2=0x7f46d413bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d413bdb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c4d0 tx=0x7f46c800c990 comp rx=0 tx=0).stop 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 shutdown_connections 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f46c006b4d0 0x7f46c006d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46d4147270 0x7f46d413bdb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46d4147810 0x7f46d4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 --2- 172.21.15.64:0/3138342929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46d40d2c10 0x7f46d413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 >> 172.21.15.64:0/3138342929 conn(0x7f46d406d4c0 msgr2=0x7f46d40b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 shutdown_connections 2024-03-07T07:20:12.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.831+0000 7f46db3a7640 1 -- 172.21.15.64:0/3138342929 wait complete. 2024-03-07T07:20:12.843 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T07:20:12.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.943+0000 7fda103fc640 1 Processor -- start 2024-03-07T07:20:12.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.943+0000 7fda103fc640 1 -- start start 2024-03-07T07:20:12.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda103fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fda08147270 0x7fda0806b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda103fc640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda08162e50 con 0x7fda080d2c10 2024-03-07T07:20:12.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda103fc640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080d1e70 con 0x7fda08147810 2024-03-07T07:20:12.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda103fc640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda08147650 con 0x7fda08147270 2024-03-07T07:20:12.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda0d997640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fda080d2c10 0x7fda08162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60184/0 (socket says 172.21.15.64:60184) 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda0d997640 1 -- 172.21.15.64:0/225174468 learned_addr learned my addr 172.21.15.64:0/225174468 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda0e198640 1 --2- 172.21.15.64:0/225174468 >> v2:172.21.15.120:3300/0 conn(0x7fda08147270 0x7fda0806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda0d196640 1 -- 172.21.15.64:0/225174468 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303596097 0 0) 0x7fda08162e50 con 0x7fda080d2c10 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.944+0000 7fda0d196640 1 -- 172.21.15.64:0/225174468 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f4003520 con 0x7fda080d2c10 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.945+0000 7fda0d196640 1 -- 172.21.15.64:0/225174468 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 868456969 0 0) 0x7fd9f4003520 con 0x7fda080d2c10 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.945+0000 7fda0e198640 1 -- 172.21.15.64:0/225174468 >> v1:172.21.15.120:6789/0 conn(0x7fda08147810 legacy=0x7fda08162740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.945+0000 7fda0e198640 1 -- 172.21.15.64:0/225174468 >> v1:172.21.15.64:6789/0 conn(0x7fda080d2c10 legacy=0x7fda08162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.945+0000 7fda0e198640 1 -- 172.21.15.64:0/225174468 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080b7520 con 0x7fda08147270 2024-03-07T07:20:12.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.945+0000 7fda0e198640 1 --2- 172.21.15.64:0/225174468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 0x7fda0806b510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3bcf9938859d7a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.946+0000 7fda0d196640 1 -- 172.21.15.64:0/225174468 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8038070 con 0x7fda08147270 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.946+0000 7fda0d196640 1 -- 172.21.15.64:0/225174468 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8031500 con 0x7fda08147270 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.946+0000 7fda0d196640 1 -- 172.21.15.64:0/225174468 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04001670 con 0x7fda08147270 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.946+0000 7fda103fc640 1 -- 172.21.15.64:0/225174468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 msgr2=0x7fda0806b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.946+0000 7fda103fc640 1 --2- 172.21.15.64:0/225174468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 0x7fda0806b510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b820 comp rx=0 tx=0).stop 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.946+0000 7fda103fc640 1 -- 172.21.15.64:0/225174468 shutdown_connections 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.946+0000 7fda103fc640 1 --2- 172.21.15.64:0/225174468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 0x7fda0806b510 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.947+0000 7fda103fc640 1 -- 172.21.15.64:0/225174468 >> 172.21.15.64:0/225174468 conn(0x7fda0806d4c0 msgr2=0x7fda08066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.947+0000 7fda103fc640 1 -- 172.21.15.64:0/225174468 shutdown_connections 2024-03-07T07:20:12.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.947+0000 7fda103fc640 1 -- 172.21.15.64:0/225174468 wait complete. 2024-03-07T07:20:12.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.948+0000 7fda103fc640 1 Processor -- start 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.948+0000 7fda103fc640 1 -- start start 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda103fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fda080d2c10 0x7fda081f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda103fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 0x7fda08140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda103fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda08147810 0x7fda080cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda103fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080d26f0 con 0x7fda08147810 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda103fc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080d2410 con 0x7fda08147270 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda103fc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda080b71d0 con 0x7fda080d2c10 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda08147810 0x7fda080cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda08147810 0x7fda080cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60394/0 (socket says 172.21.15.64:60394) 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 -- 172.21.15.64:0/1428524571 learned_addr learned my addr 172.21.15.64:0/1428524571 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e198640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fda080d2c10 0x7fda081f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0d997640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 0x7fda08140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 -- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fda080d2c10 msgr2=0x7fda081f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fda080d2c10 0x7fda081f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 -- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 msgr2=0x7fda08140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 0x7fda08140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 -- 172.21.15.64:0/1428524571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda081f4cd0 con 0x7fda08147810 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda0e999640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda08147810 0x7fda080cd600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c6b0 tx=0x7fda0000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fd9feffd640 1 -- 172.21.15.64:0/1428524571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00003b70 con 0x7fda08147810 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fd9feffd640 1 -- 172.21.15.64:0/1428524571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda00003d30 con 0x7fda08147810 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.950+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda080cbac0 con 0x7fda08147810 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.951+0000 7fd9feffd640 1 -- 172.21.15.64:0/1428524571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00005310 con 0x7fda08147810 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.951+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda080cc040 con 0x7fda08147810 2024-03-07T07:20:12.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.951+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda08062c90 con 0x7fda08147810 2024-03-07T07:20:12.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.952+0000 7fd9feffd640 1 -- 172.21.15.64:0/1428524571 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fda000054b0 con 0x7fda08147810 2024-03-07T07:20:12.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.952+0000 7fd9feffd640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:12.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.952+0000 7fd9feffd640 1 -- 172.21.15.64:0/1428524571 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda00089ff0 con 0x7fda08147810 2024-03-07T07:20:12.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.953+0000 7fda0e198640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:12.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.953+0000 7fda0e198640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8035000 tx=0x7fd9f8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:12.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:12.957+0000 7fd9feffd640 1 -- 172.21.15.64:0/1428524571 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fda081f4cd0 con 0x7fda08147810 2024-03-07T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.196+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fda08071cc0 con 0x7fda08147810 2024-03-07T07:20:13.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.196+0000 7fd9feffd640 1 -- 172.21.15.64:0/1428524571 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7fda00075e20 con 0x7fda08147810 2024-03-07T07:20:13.198 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd9ec06b4d0 msgr2=0x7fd9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8035000 tx=0x7fd9f8008030 comp rx=0 tx=0).stop 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda08147810 msgr2=0x7fda080cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda08147810 0x7fda080cd600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c6b0 tx=0x7fda0000cb70 comp rx=0 tx=0).stop 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 shutdown_connections 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda08147810 0x7fda080cd600 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:13.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda08147270 0x7fda08140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 --2- 172.21.15.64:0/1428524571 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fda080d2c10 0x7fda081f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 >> 172.21.15.64:0/1428524571 conn(0x7fda0806d4c0 msgr2=0x7fda080b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 shutdown_connections 2024-03-07T07:20:13.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:13.199+0000 7fda103fc640 1 -- 172.21.15.64:0/1428524571 wait complete. 2024-03-07T07:20:13.210 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-07T07:20:13.210 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T07:20:13.210 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-03-07T07:20:13.210 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-07T07:20:13.210 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T07:20:13.210 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-07T07:20:13.211 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-07T07:20:13.211 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T07:20:13.211 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T07:20:13.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.211 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-03-07T07:20:13.226 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42663sec preferred_lft 42663sec 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 2a:12:30:b0:4b:5a brd ff:ff:ff:ff:ff:ff 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:20:13.227 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::48a3:bdff:feea:4c56/64 scope link 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether fa:1a:1f:08:56:b8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::f81a:1fff:fe08:56b8/64 scope link 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 2a:12:30:b0:4b:5a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::2812:30ff:feb0:4b5a/64 scope link 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-03-07T07:20:13.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.228 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T07:20:13.291 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-03-07T07:20:13.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.293 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-03-07T07:20:13.347 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-03-07T07:20:13.347 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T07:20:13.347 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T07:20:13.347 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T07:20:13.347 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T07:20:13.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.348 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec bash -c ' 2024-03-07T07:20:13.348 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-03-07T07:20:13.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T07:20:13.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T07:20:13.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T07:20:13.348 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-03-07T07:20:13.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.442 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-03-07T07:20:13.456 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-07T07:20:13.457 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T07:20:13.457 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T07:20:13.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-03-07T07:20:13.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:20:13.524 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-07T07:20:13.578 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-07T07:20:13.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T07:20:13.578 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-07T07:20:13.631 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-07T07:20:13.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.632 DEBUG:teuthology.orchestra.run.smithi064:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:20:13.718 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:20:13.718 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:20:13.725 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T07:20:13.725 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T07:20:13.725 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: resolved to: "172.21.15.64:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-03-07T07:20:13.726 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: trying mount with new device syntax: 0@6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3.cephfs=/ 2024-03-07T07:20:13.726 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.64:3300/172.21.15.120:3300/172.21.15.120:3301" will pass to kernel 2024-03-07T07:20:13.726 INFO:teuthology.orchestra.run.smithi064.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-07T07:20:13.726 INFO:teuthology.orchestra.run.smithi064.stdout: You just mounted a file system that supports labels which does not 2024-03-07T07:20:13.726 INFO:teuthology.orchestra.run.smithi064.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T07:20:13.726 INFO:teuthology.orchestra.run.smithi064.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T07:20:13.727 INFO:teuthology.orchestra.run.smithi064.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T07:20:13.729 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T07:20:13.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.729 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-07T07:20:13.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.756 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T07:20:13.816 INFO:teuthology.orchestra.run.smithi064.stdout:client5280 2024-03-07T07:20:13.817 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:20:13.817 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:20:13.872 DEBUG:teuthology.orchestra.run.smithi064:> mktemp -d --suffix=xfstests-dev 2024-03-07T07:20:13.927 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.9XyAfeyTXrxfstests-dev 2024-03-07T07:20:13.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.927 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.9XyAfeyTXrxfstests-dev) 2024-03-07T07:20:13.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:13.995 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.9XyAfeyTXrxfstests-dev) 2024-03-07T07:20:14.052 INFO:teuthology.orchestra.run.smithi064.stderr:Cloning into '/tmp/tmp.9XyAfeyTXrxfstests-dev'... 2024-03-07T07:20:14.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:14.878 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-03-07T07:20:14.936 DEBUG:teuthology.orchestra.run.smithi064:> mktemp -d --suffix=test 2024-03-07T07:20:14.991 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.lAL4aSV2w3test 2024-03-07T07:20:14.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:14.991 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-03-07T07:20:15.048 DEBUG:teuthology.orchestra.run.smithi064:> mktemp -d --suffix=scratch 2024-03-07T07:20:15.103 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.KgIAmWnlfEscratch 2024-03-07T07:20:15.103 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-03-07T07:20:15.103 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-03-07T07:20:15.768 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:03:20 ago on Thu 07 Mar 2024 07:16:55 AM UTC. 2024-03-07T07:20:16.082 INFO:teuthology.orchestra.run.smithi064.stderr:Modular dependency problems: 2024-03-07T07:20:16.082 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:20:16.082 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 1: conflicting requests 2024-03-07T07:20:16.082 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.082 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 2: conflicting requests 2024-03-07T07:20:16.082 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.082 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 3: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 4: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 5: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 6: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 7: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 8: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 9: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 10: conflicting requests 2024-03-07T07:20:16.083 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 11: conflicting requests 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 12: conflicting requests 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 13: conflicting requests 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 14: conflicting requests 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 15: conflicting requests 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 16: conflicting requests 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.084 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 17: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 18: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 19: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 20: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 21: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 22: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 23: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 24: conflicting requests 2024-03-07T07:20:16.085 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 25: conflicting requests 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 26: conflicting requests 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 27: conflicting requests 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 28: conflicting requests 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 29: conflicting requests 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 30: conflicting requests 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 31: conflicting requests 2024-03-07T07:20:16.086 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 32: conflicting requests 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 33: conflicting requests 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 34: conflicting requests 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 35: conflicting requests 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 36: conflicting requests 2024-03-07T07:20:16.087 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 37: conflicting requests 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 39: conflicting requests 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.088 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.089 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.090 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.091 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.091 INFO:teuthology.orchestra.run.smithi064.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T07:20:16.091 INFO:teuthology.orchestra.run.smithi064.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-03-07T07:20:16.092 INFO:teuthology.orchestra.run.smithi064.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T07:20:16.092 INFO:teuthology.orchestra.run.smithi064.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-03-07T07:20:16.092 INFO:teuthology.orchestra.run.smithi064.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-03-07T07:20:16.092 INFO:teuthology.orchestra.run.smithi064.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-03-07T07:20:16.092 INFO:teuthology.orchestra.run.smithi064.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-03-07T07:20:16.094 INFO:teuthology.orchestra.run.smithi064.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-03-07T07:20:16.095 INFO:teuthology.orchestra.run.smithi064.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-03-07T07:20:16.097 INFO:teuthology.orchestra.run.smithi064.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2024-03-07T07:20:16.100 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-03-07T07:20:16.101 INFO:teuthology.orchestra.run.smithi064.stdout:Package libaio-devel-0.3.112-1.el8.x86_64 is already installed. 2024-03-07T07:20:16.103 INFO:teuthology.orchestra.run.smithi064.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-03-07T07:20:16.104 INFO:teuthology.orchestra.run.smithi064.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-03-07T07:20:16.105 INFO:teuthology.orchestra.run.smithi064.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-03-07T07:20:16.106 INFO:teuthology.orchestra.run.smithi064.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-03-07T07:20:16.107 INFO:teuthology.orchestra.run.smithi064.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-03-07T07:20:16.108 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-03-07T07:20:16.109 INFO:teuthology.orchestra.run.smithi064.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-03-07T07:20:16.110 INFO:teuthology.orchestra.run.smithi064.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-03-07T07:20:16.113 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-03-07T07:20:16.194 INFO:teuthology.orchestra.run.smithi064.stderr:Error: 2024-03-07T07:20:16.194 INFO:teuthology.orchestra.run.smithi064.stderr: Problem: package python3-libs-3.9.18-3.el9.x86_64 from baseos requires libreadline.so.8()(64bit), but none of the providers can be installed 2024-03-07T07:20:16.195 INFO:teuthology.orchestra.run.smithi064.stderr: - cannot install both readline-7.0-10.el8.x86_64 from CentOS-BaseOS and readline-8.1-4.el9.x86_64 from @System 2024-03-07T07:20:16.195 INFO:teuthology.orchestra.run.smithi064.stderr: - cannot install both readline-8.1-4.el9.x86_64 from baseos and readline-7.0-10.el8.x86_64 from CentOS-BaseOS 2024-03-07T07:20:16.195 INFO:teuthology.orchestra.run.smithi064.stderr: - package udftools-2.3-2.el8.x86_64 from CentOS-AppStream requires libreadline.so.7()(64bit), but none of the providers can be installed 2024-03-07T07:20:16.195 INFO:teuthology.orchestra.run.smithi064.stderr: - package python3-3.9.18-3.el9.x86_64 from baseos requires python3-libs(x86-64) = 3.9.18-3.el9, but none of the providers can be installed 2024-03-07T07:20:16.195 INFO:teuthology.orchestra.run.smithi064.stderr: - cannot install the best candidate for the job 2024-03-07T07:20:16.195 INFO:teuthology.orchestra.run.smithi064.stdout:(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) 2024-03-07T07:20:16.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:20:16.233 INFO:tasks.cephfs_test_runner:test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) ... ERROR 2024-03-07T07:20:16.233 INFO:tasks.cephfs_test_runner: 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner:ERROR: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/xfstests_dev.py", line 29, in setUp 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: self.prepare_xfstests_devs() 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/xfstests_dev.py", line 44, in prepare_xfstests_devs 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: self.install_deps() 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/xfstests_dev.py", line 252, in install_deps 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: self.mount_a.client_remote.run(args=args, omit_sudo=False) 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-07T07:20:16.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi064 with status 1: 'sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs' 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner:Ran 1 test in 31.628s 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner: 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner:ERROR: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-03-07T07:20:16.235 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/xfstests_dev.py", line 29, in setUp 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: self.prepare_xfstests_devs() 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/xfstests_dev.py", line 44, in prepare_xfstests_devs 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: self.install_deps() 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/xfstests_dev.py", line 252, in install_deps 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: self.mount_a.client_remote.run(args=args, omit_sudo=False) 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi064 with status 1: 'sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs' 2024-03-07T07:20:16.236 INFO:tasks.cephfs_test_runner: 2024-03-07T07:20:16.237 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-03-07T07:20:16.465 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=cad744c4d5c7483f862069efe14b291f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-03-07T07:20:16.467 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-07T07:20:16.475 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-07T07:20:16.483 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-07T07:20:16.483 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:20:16.483 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:20:16.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:20:16.495 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:20:16.501 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:20:16.501 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:20:16.595 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:20:16.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab383a7640 1 Processor -- start 2024-03-07T07:20:16.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab383a7640 1 -- start start 2024-03-07T07:20:16.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab383a7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b5dd0 0x7fab30144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:16.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab383a7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab300b43f0 0x7fab30162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:16.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab383a7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab300b4bd0 con 0x7fab30162ab0 2024-03-07T07:20:16.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab383a7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab30141700 con 0x7fab300b43f0 2024-03-07T07:20:16.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab383a7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab30161e00 con 0x7fab300b5dd0 2024-03-07T07:20:16.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab35942640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab300b43f0 0x7fab30162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:16.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab35942640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab300b43f0 0x7fab30162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60400/0 (socket says 172.21.15.64:60400) 2024-03-07T07:20:16.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.739+0000 7fab35942640 1 -- 172.21.15.64:0/492494314 learned_addr learned my addr 172.21.15.64:0/492494314 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:16.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.740+0000 7fab36143640 1 --2- 172.21.15.64:0/492494314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b5dd0 0x7fab30144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:16.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.740+0000 7fab35942640 1 -- 172.21.15.64:0/492494314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b5dd0 msgr2=0x7fab30144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:16.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.740+0000 7fab35942640 1 --2- 172.21.15.64:0/492494314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b5dd0 0x7fab30144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:16.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.740+0000 7fab35942640 1 -- 172.21.15.64:0/492494314 >> v1:172.21.15.120:6789/0 conn(0x7fab30162ab0 legacy=0x7fab30071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:16.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.740+0000 7fab35942640 1 -- 172.21.15.64:0/492494314 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab300b4eb0 con 0x7fab300b43f0 2024-03-07T07:20:16.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab35942640 1 --2- 172.21.15.64:0/492494314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab300b43f0 0x7fab30162570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fab30161f80 tx=0x7fab2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e24f74d413c95e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:16.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab35141640 1 -- 172.21.15.64:0/492494314 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24035020 con 0x7fab300b43f0 2024-03-07T07:20:16.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab35141640 1 -- 172.21.15.64:0/492494314 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab24031640 con 0x7fab300b43f0 2024-03-07T07:20:16.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab35141640 1 -- 172.21.15.64:0/492494314 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2402d040 con 0x7fab300b43f0 2024-03-07T07:20:16.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 -- 172.21.15.64:0/492494314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab300b43f0 msgr2=0x7fab30162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:16.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 --2- 172.21.15.64:0/492494314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab300b43f0 0x7fab30162570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fab30161f80 tx=0x7fab2402b8f0 comp rx=0 tx=0).stop 2024-03-07T07:20:16.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 -- 172.21.15.64:0/492494314 shutdown_connections 2024-03-07T07:20:16.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 --2- 172.21.15.64:0/492494314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab300b43f0 0x7fab30162570 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:16.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 --2- 172.21.15.64:0/492494314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b5dd0 0x7fab30144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:16.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 -- 172.21.15.64:0/492494314 >> 172.21.15.64:0/492494314 conn(0x7fab300d0e70 msgr2=0x7fab300d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:16.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 -- 172.21.15.64:0/492494314 shutdown_connections 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.741+0000 7fab383a7640 1 -- 172.21.15.64:0/492494314 wait complete. 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.742+0000 7fab383a7640 1 Processor -- start 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab383a7640 1 -- start start 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab383a7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b43f0 0x7fab301f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab383a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab300b5dd0 0x7fab3015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab383a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab301f4b00 0x7fab300d2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab383a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300b61b0 con 0x7fab301f4b00 2024-03-07T07:20:16.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab383a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30162050 con 0x7fab300b5dd0 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab383a7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab300b4870 con 0x7fab300b43f0 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab36944640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab301f4b00 0x7fab300d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab36944640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab301f4b00 0x7fab300d2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60408/0 (socket says 172.21.15.64:60408) 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.743+0000 7fab36944640 1 -- 172.21.15.64:0/591760453 learned_addr learned my addr 172.21.15.64:0/591760453 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab35942640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab300b5dd0 0x7fab3015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab36143640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b43f0 0x7fab301f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab36944640 1 -- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b43f0 msgr2=0x7fab301f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab36944640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b43f0 0x7fab301f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:16.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab36944640 1 -- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab300b5dd0 msgr2=0x7fab3015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab36944640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab300b5dd0 0x7fab3015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab36944640 1 -- 172.21.15.64:0/591760453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30142970 con 0x7fab301f4b00 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab36944640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab301f4b00 0x7fab300d2810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c6e0 tx=0x7fab2000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab2effd640 1 -- 172.21.15.64:0/591760453 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20003b70 con 0x7fab301f4b00 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab2effd640 1 -- 172.21.15.64:0/591760453 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab20003d30 con 0x7fab301f4b00 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab30142be0 con 0x7fab301f4b00 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab2effd640 1 -- 172.21.15.64:0/591760453 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20005310 con 0x7fab301f4b00 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.744+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab301431a0 con 0x7fab301f4b00 2024-03-07T07:20:16.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.745+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab30062c90 con 0x7fab301f4b00 2024-03-07T07:20:16.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.745+0000 7fab2effd640 1 -- 172.21.15.64:0/591760453 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fab200054b0 con 0x7fab301f4b00 2024-03-07T07:20:16.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.746+0000 7fab2effd640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab1c06b4d0 0x7fab1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:16.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.746+0000 7fab2effd640 1 -- 172.21.15.64:0/591760453 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab20089fb0 con 0x7fab301f4b00 2024-03-07T07:20:16.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.746+0000 7fab36143640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab1c06b4d0 0x7fab1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:16.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.747+0000 7fab36143640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab1c06b4d0 0x7fab1c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab28006fd0 tx=0x7fab28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:16.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:16.750+0000 7fab2effd640 1 -- 172.21.15.64:0/591760453 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab20058ec0 con 0x7fab301f4b00 2024-03-07T07:20:17.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.045+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab301f1720 con 0x7fab301f4b00 2024-03-07T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.047+0000 7fab2effd640 1 -- 172.21.15.64:0/591760453 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16633 (secure 0 0 0) 0x7fab200587c0 con 0x7fab301f4b00 2024-03-07T07:20:17.047 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:17.048 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:20:09.012362+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:20:05.640176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T07:20:06.344694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab1c06b4d0 msgr2=0x7fab1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab1c06b4d0 0x7fab1c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab28006fd0 tx=0x7fab28008040 comp rx=0 tx=0).stop 2024-03-07T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab301f4b00 msgr2=0x7fab300d2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab301f4b00 0x7fab300d2810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c6e0 tx=0x7fab2000cba0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 shutdown_connections 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab1c06b4d0 0x7fab1c06d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab301f4b00 0x7fab300d2810 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab300b5dd0 0x7fab3015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 --2- 172.21.15.64:0/591760453 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab300b43f0 0x7fab301f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 >> 172.21.15.64:0/591760453 conn(0x7fab300d0e70 msgr2=0x7fab3015e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 shutdown_connections 2024-03-07T07:20:17.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.049+0000 7fab383a7640 1 -- 172.21.15.64:0/591760453 wait complete. 2024-03-07T07:20:17.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T07:19:04.222822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T07:20:05.640176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T07:20:06.344694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T07:20:17.064 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T07:20:17.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.168+0000 7facc5234640 1 Processor -- start 2024-03-07T07:20:17.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.169+0000 7facc5234640 1 -- start start 2024-03-07T07:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facc5234640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7facc015fbe0 0x7facc00cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facc5234640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc00d1090 con 0x7facc009b0e0 2024-03-07T07:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facc5234640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc00d1370 con 0x7facc015f800 2024-03-07T07:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facc5234640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc00d1650 con 0x7facc015fbe0 2024-03-07T07:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbe59c640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7facc009b0e0 0x7facc00ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60192/0 (socket says 172.21.15.64:60192) 2024-03-07T07:20:17.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbe59c640 1 -- 172.21.15.64:0/33897994 learned_addr learned my addr 172.21.15.64:0/33897994 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344713033 0 0) 0x7facc00d1090 con 0x7facc009b0e0 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faca8003520 con 0x7facc009b0e0 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3071635286 0 0) 0x7faca8003520 con 0x7facc009b0e0 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 >> v2:172.21.15.120:3300/0 conn(0x7facc015fbe0 msgr2=0x7facc00cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 --2- 172.21.15.64:0/33897994 >> v2:172.21.15.120:3300/0 conn(0x7facc015fbe0 0x7facc00cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 >> v1:172.21.15.120:6789/0 conn(0x7facc015f800 legacy=0x7facc009a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00d1930 con 0x7facc009b0e0 2024-03-07T07:20:17.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7facb4001670 con 0x7facc009b0e0 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 >> v1:172.21.15.64:6789/0 conn(0x7facc009b0e0 legacy=0x7facc00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.170+0000 7facbdd9b640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca80039b0 0x7faca8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbdd9b640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca8004800 0x7faca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbdd9b640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca80052e0 0x7faca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00d1650 con 0x7faca80039b0 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca800c1e0 con 0x7faca80052e0 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca800c3e0 con 0x7faca8004800 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbed9d640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca80039b0 0x7faca8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbf59e640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca80052e0 0x7faca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbe59c640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca8004800 0x7faca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbed9d640 1 -- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca8004800 msgr2=0x7faca8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbed9d640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca8004800 0x7faca8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbed9d640 1 -- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca80052e0 msgr2=0x7faca800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbed9d640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca80052e0 0x7faca800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbed9d640 1 -- 172.21.15.64:0/33897994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8003520 con 0x7faca80039b0 2024-03-07T07:20:17.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.171+0000 7facbf59e640 1 -- 172.21.15.64:0/33897994 reap_dead start 2024-03-07T07:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.172+0000 7facbed9d640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca80039b0 0x7faca8003db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7facb8001620 tx=0x7faca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb8ede04d321a5e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.172+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8001220 con 0x7faca80039b0 2024-03-07T07:20:17.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.172+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb8005070 con 0x7faca80039b0 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.172+0000 7facbdd9b640 1 -- 172.21.15.64:0/33897994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8005480 con 0x7faca80039b0 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.172+0000 7facc5234640 1 -- 172.21.15.64:0/33897994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca80039b0 msgr2=0x7faca8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.173+0000 7facc5234640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca80039b0 0x7faca8003db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7facb8001620 tx=0x7faca8005830 comp rx=0 tx=0).stop 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.173+0000 7facc5234640 1 -- 172.21.15.64:0/33897994 shutdown_connections 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.173+0000 7facc5234640 1 --2- 172.21.15.64:0/33897994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faca80039b0 0x7faca8003db0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.173+0000 7facc5234640 1 -- 172.21.15.64:0/33897994 >> 172.21.15.64:0/33897994 conn(0x7facc00b3b90 msgr2=0x7facc00b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.173+0000 7facc5234640 1 -- 172.21.15.64:0/33897994 shutdown_connections 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.173+0000 7facc5234640 1 -- 172.21.15.64:0/33897994 wait complete. 2024-03-07T07:20:17.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.174+0000 7facc5234640 1 Processor -- start 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.174+0000 7facc5234640 1 -- start start 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facc5234640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca80039b0 0x7facc01f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facc5234640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc00b6460 0x7facc00cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facc5234640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc00cdfb0 0x7facc00b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facc5234640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00b0fb0 con 0x7facc00cdfb0 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facc5234640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00b15e0 con 0x7facc00b6460 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facc5234640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00b1130 con 0x7faca80039b0 2024-03-07T07:20:17.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc00cdfb0 0x7facc00b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbed9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca80039b0 0x7facc01f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbe59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc00b6460 0x7facc00cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc00cdfb0 0x7facc00b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60436/0 (socket says 172.21.15.64:60436) 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbf59e640 1 -- 172.21.15.64:0/903212012 learned_addr learned my addr 172.21.15.64:0/903212012 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbed9d640 1 -- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc00b6460 msgr2=0x7facc00cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbed9d640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc00b6460 0x7facc00cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbed9d640 1 -- 172.21.15.64:0/903212012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc00cdfb0 msgr2=0x7facc00b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbed9d640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc00cdfb0 0x7facc00b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbed9d640 1 -- 172.21.15.64:0/903212012 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00b68f0 con 0x7faca80039b0 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbf59e640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc00cdfb0 0x7facc00b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.175+0000 7facbed9d640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca80039b0 0x7facc01f1d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7facb8030000 tx=0x7facb8000fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.176+0000 7faca77fe640 1 -- 172.21.15.64:0/903212012 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8042020 con 0x7faca80039b0 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.176+0000 7faca77fe640 1 -- 172.21.15.64:0/903212012 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb8039040 con 0x7faca80039b0 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.176+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc00b6bd0 con 0x7faca80039b0 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.176+0000 7faca77fe640 1 -- 172.21.15.64:0/903212012 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8003a30 con 0x7faca80039b0 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.176+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc00b7190 con 0x7faca80039b0 2024-03-07T07:20:17.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.177+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc00ce3c0 con 0x7faca80039b0 2024-03-07T07:20:17.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.177+0000 7faca77fe640 1 -- 172.21.15.64:0/903212012 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7facb8045020 con 0x7faca80039b0 2024-03-07T07:20:17.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.178+0000 7faca77fe640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7facac06b4d0 0x7facac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.178+0000 7faca77fe640 1 -- 172.21.15.64:0/903212012 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facb80abc50 con 0x7faca80039b0 2024-03-07T07:20:17.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.178+0000 7facbe59c640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7facac06b4d0 0x7facac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.178+0000 7facbe59c640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7facac06b4d0 0x7facac06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7facc00673c0 tx=0x7facb4001c30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.187+0000 7faca77fe640 1 -- 172.21.15.64:0/903212012 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7facc00ce3c0 con 0x7faca80039b0 2024-03-07T07:20:17.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.366+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facc00b73d0 con 0x7faca80039b0 2024-03-07T07:20:17.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.367+0000 7faca77fe640 1 -- 172.21.15.64:0/903212012 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7facb807a360 con 0x7faca80039b0 2024-03-07T07:20:17.367 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-03-07T07:20:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7facac06b4d0 msgr2=0x7facac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7facac06b4d0 0x7facac06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7facc00673c0 tx=0x7facb4001c30 comp rx=0 tx=0).stop 2024-03-07T07:20:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca80039b0 msgr2=0x7facc01f1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca80039b0 0x7facc01f1d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7facb8030000 tx=0x7facb8000fb0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 shutdown_connections 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7facac06b4d0 0x7facac06d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7facc00cdfb0 0x7facc00b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7facc00b6460 0x7facc00cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 --2- 172.21.15.64:0/903212012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faca80039b0 0x7facc01f1d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 >> 172.21.15.64:0/903212012 conn(0x7facc00b3b90 msgr2=0x7facc009a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 shutdown_connections 2024-03-07T07:20:17.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.369+0000 7facc5234640 1 -- 172.21.15.64:0/903212012 wait complete. 2024-03-07T07:20:17.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T07:20:17.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.489+0000 7fab5301a640 1 Processor -- start 2024-03-07T07:20:17.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.489+0000 7fab5301a640 1 -- start start 2024-03-07T07:20:17.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab5301a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab5301a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab4c1629c0 0x7fab4c0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab5301a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4c0b34d0 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab5301a640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4c161ee0 con 0x7fab4c1629c0 2024-03-07T07:20:17.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab5301a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c1621c0 con 0x7fab4c15dff0 2024-03-07T07:20:17.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab4bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab4c1629c0 0x7fab4c0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab4bfff640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fab4c1629c0 0x7fab4c0b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60450/0 (socket says 172.21.15.64:60450) 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab4bfff640 1 -- 172.21.15.64:0/3708701150 learned_addr learned my addr 172.21.15.64:0/3708701150 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab50db6640 1 --2- 172.21.15.64:0/3708701150 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab4bfff640 1 -- 172.21.15.64:0/3708701150 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 msgr2=0x7fab4c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab4bfff640 1 --2- 172.21.15.64:0/3708701150 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab4bfff640 1 -- 172.21.15.64:0/3708701150 >> v1:172.21.15.120:6789/0 conn(0x7fab4c0b2fc0 legacy=0x7fab4c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab4bfff640 1 -- 172.21.15.64:0/3708701150 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c1624a0 con 0x7fab4c1629c0 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.490+0000 7fab50db6640 1 --2- 172.21.15.64:0/3708701150 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:17.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.491+0000 7fab4bfff640 1 --2- 172.21.15.64:0/3708701150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c1629c0 0x7fab4c0b3df0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fab38009870 tx=0x7fab3802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cbaed1b55661ff server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.491+0000 7fab4b7fe640 1 -- 172.21.15.64:0/3708701150 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38034070 con 0x7fab4c1629c0 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.491+0000 7fab4b7fe640 1 -- 172.21.15.64:0/3708701150 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38030840 con 0x7fab4c1629c0 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.491+0000 7fab4b7fe640 1 -- 172.21.15.64:0/3708701150 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38030c10 con 0x7fab4c1629c0 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 -- 172.21.15.64:0/3708701150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c1629c0 msgr2=0x7fab4c0b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 --2- 172.21.15.64:0/3708701150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c1629c0 0x7fab4c0b3df0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fab38009870 tx=0x7fab3802b980 comp rx=0 tx=0).stop 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 -- 172.21.15.64:0/3708701150 shutdown_connections 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 --2- 172.21.15.64:0/3708701150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c1629c0 0x7fab4c0b3df0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 --2- 172.21.15.64:0/3708701150 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 -- 172.21.15.64:0/3708701150 >> 172.21.15.64:0/3708701150 conn(0x7fab4c06b040 msgr2=0x7fab4c06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:17.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 -- 172.21.15.64:0/3708701150 shutdown_connections 2024-03-07T07:20:17.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.492+0000 7fab5301a640 1 -- 172.21.15.64:0/3708701150 wait complete. 2024-03-07T07:20:17.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.493+0000 7fab5301a640 1 Processor -- start 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.493+0000 7fab5301a640 1 -- start start 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c0b2fc0 0x7fab4c147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab4c1629c0 0x7fab4c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c15e550 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c15e3d0 con 0x7fab4c1629c0 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c0b4330 con 0x7fab4c15dff0 2024-03-07T07:20:17.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c0b2fc0 0x7fab4c147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c0b2fc0 0x7fab4c147820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60460/0 (socket says 172.21.15.64:60460) 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 -- 172.21.15.64:0/175241110 learned_addr learned my addr 172.21.15.64:0/175241110 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab4bfff640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab515b7640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab4c1629c0 0x7fab4c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 -- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 msgr2=0x7fab4c15f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c15f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 -- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab4c1629c0 msgr2=0x7fab4c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab4c1629c0 0x7fab4c15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 -- 172.21.15.64:0/175241110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c147d60 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab50db6640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c0b2fc0 0x7fab4c147820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fab4000e730 tx=0x7fab4000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab497fa640 1 -- 172.21.15.64:0/175241110 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000f040 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab497fa640 1 -- 172.21.15.64:0/175241110 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40004230 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c15e8e0 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab497fa640 1 -- 172.21.15.64:0/175241110 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000ee40 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.494+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c15eea0 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.495+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4c062c90 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.496+0000 7fab497fa640 1 -- 172.21.15.64:0/175241110 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fab4000c6d0 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.496+0000 7fab497fa640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab3006b4d0 0x7fab3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.497+0000 7fab497fa640 1 -- 172.21.15.64:0/175241110 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab40089e40 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.497+0000 7fab4bfff640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab3006b4d0 0x7fab3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.497+0000 7fab4bfff640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab3006b4d0 0x7fab3006d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab38036000 tx=0x7fab3803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.501+0000 7fab497fa640 1 -- 172.21.15.64:0/175241110 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab40017020 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.679+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab4c0b38f0 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.680+0000 7fab497fa640 1 -- 172.21.15.64:0/175241110 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab40058700 con 0x7fab4c0b2fc0 2024-03-07T07:20:17.680 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-03-07T07:20:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab3006b4d0 msgr2=0x7fab3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab3006b4d0 0x7fab3006d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab38036000 tx=0x7fab3803b040 comp rx=0 tx=0).stop 2024-03-07T07:20:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c0b2fc0 msgr2=0x7fab4c147820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c0b2fc0 0x7fab4c147820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fab4000e730 tx=0x7fab4000ebf0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 shutdown_connections 2024-03-07T07:20:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fab3006b4d0 0x7fab3006d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab4c1629c0 0x7fab4c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fab4c15dff0 0x7fab4c15f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 --2- 172.21.15.64:0/175241110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab4c0b2fc0 0x7fab4c147820 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 >> 172.21.15.64:0/175241110 conn(0x7fab4c06b040 msgr2=0x7fab4c070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 shutdown_connections 2024-03-07T07:20:17.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.682+0000 7fab5301a640 1 -- 172.21.15.64:0/175241110 wait complete. 2024-03-07T07:20:17.693 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T07:20:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.800+0000 7fb280d67640 1 Processor -- start 2024-03-07T07:20:17.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.800+0000 7fb280d67640 1 -- start start 2024-03-07T07:20:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb280d67640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c09b0e0 0x7fb27c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb280d67640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb27c15fbe0 0x7fb27c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb280d67640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27c0d1090 con 0x7fb27c15f800 2024-03-07T07:20:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb280d67640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27c0d1370 con 0x7fb27c15fbe0 2024-03-07T07:20:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb280d67640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c0d1650 con 0x7fb27c09b0e0 2024-03-07T07:20:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb279d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c09b0e0 0x7fb27c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb27ad9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb27c15fbe0 0x7fb27c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb27a59c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb27c15f800 0x7fb27c09a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:40170/0 (socket says 172.21.15.64:40170) 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb279d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c09b0e0 0x7fb27c0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:59138/0 (socket says 172.21.15.64:59138) 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.801+0000 7fb27a59c640 1 -- 172.21.15.64:0/2253919322 learned_addr learned my addr 172.21.15.64:0/2253919322 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.802+0000 7fb27ad9d640 1 -- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c09b0e0 msgr2=0x7fb27c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.802+0000 7fb27ad9d640 1 --2- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c09b0e0 0x7fb27c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.802+0000 7fb27ad9d640 1 -- 172.21.15.64:0/2253919322 >> v1:172.21.15.120:6789/0 conn(0x7fb27c15f800 legacy=0x7fb27c09a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.802+0000 7fb27ad9d640 1 -- 172.21.15.64:0/2253919322 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c0d1930 con 0x7fb27c15fbe0 2024-03-07T07:20:17.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.802+0000 7fb279d9b640 1 --2- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c09b0e0 0x7fb27c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:17.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.802+0000 7fb27ad9d640 1 --2- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c15fbe0 0x7fb27c0cd190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb274009d90 tx=0x7fb27402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eaa0e8dc29cfb48 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb27959a640 1 -- 172.21.15.64:0/2253919322 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274039070 con 0x7fb27c15fbe0 2024-03-07T07:20:17.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb27959a640 1 -- 172.21.15.64:0/2253919322 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27402cc50 con 0x7fb27c15fbe0 2024-03-07T07:20:17.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb27959a640 1 -- 172.21.15.64:0/2253919322 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274034440 con 0x7fb27c15fbe0 2024-03-07T07:20:17.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb280d67640 1 -- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c15fbe0 msgr2=0x7fb27c0cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb280d67640 1 --2- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c15fbe0 0x7fb27c0cd190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb274009d90 tx=0x7fb27402bea0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb280d67640 1 -- 172.21.15.64:0/2253919322 shutdown_connections 2024-03-07T07:20:17.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb280d67640 1 --2- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c15fbe0 0x7fb27c0cd190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb280d67640 1 --2- 172.21.15.64:0/2253919322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c09b0e0 0x7fb27c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb280d67640 1 -- 172.21.15.64:0/2253919322 >> 172.21.15.64:0/2253919322 conn(0x7fb27c0b3b90 msgr2=0x7fb27c0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:17.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.803+0000 7fb280d67640 1 -- 172.21.15.64:0/2253919322 shutdown_connections 2024-03-07T07:20:17.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.804+0000 7fb280d67640 1 -- 172.21.15.64:0/2253919322 wait complete. 2024-03-07T07:20:17.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.805+0000 7fb280d67640 1 Processor -- start 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.805+0000 7fb280d67640 1 -- start start 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb280d67640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c09b0e0 0x7fb27c14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb280d67640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb27c15f800 0x7fb27c1f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb280d67640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c15fbe0 0x7fb27c0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb280d67640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c0b0fb0 con 0x7fb27c15f800 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb280d67640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c0b15e0 con 0x7fb27c09b0e0 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb280d67640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c0b1130 con 0x7fb27c15fbe0 2024-03-07T07:20:17.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb279d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb27c15f800 0x7fb27c1f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c09b0e0 0x7fb27c14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb279d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb27c15f800 0x7fb27c1f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60470/0 (socket says 172.21.15.64:60470) 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb279d9b640 1 -- 172.21.15.64:0/4129495807 learned_addr learned my addr 172.21.15.64:0/4129495807 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c09b0e0 0x7fb27c14e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:37130/0 (socket says 172.21.15.64:37130) 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27ad9d640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c15fbe0 0x7fb27c0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 -- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c15fbe0 msgr2=0x7fb27c0b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c15fbe0 0x7fb27c0b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 -- 172.21.15.64:0/4129495807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb27c15f800 msgr2=0x7fb27c1f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb27c15f800 0x7fb27c1f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 -- 172.21.15.64:0/4129495807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c0c9150 con 0x7fb27c09b0e0 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb27a59c640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c09b0e0 0x7fb27c14e4d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c370 tx=0x7fb26c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.807+0000 7fb26affd640 1 -- 172.21.15.64:0/4129495807 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c017070 con 0x7fb27c09b0e0 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.808+0000 7fb26affd640 1 -- 172.21.15.64:0/4129495807 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c005ce0 con 0x7fb27c09b0e0 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.808+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c1f5d20 con 0x7fb27c09b0e0 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.808+0000 7fb26affd640 1 -- 172.21.15.64:0/4129495807 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c007310 con 0x7fb27c09b0e0 2024-03-07T07:20:17.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.808+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c0c93c0 con 0x7fb27c09b0e0 2024-03-07T07:20:17.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.809+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c062c90 con 0x7fb27c09b0e0 2024-03-07T07:20:17.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.809+0000 7fb26affd640 1 -- 172.21.15.64:0/4129495807 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb26c01a070 con 0x7fb27c09b0e0 2024-03-07T07:20:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.809+0000 7fb26affd640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.810+0000 7fb26affd640 1 -- 172.21.15.64:0/4129495807 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb26c0894d0 con 0x7fb27c09b0e0 2024-03-07T07:20:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.810+0000 7fb279d9b640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:17.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.810+0000 7fb279d9b640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2700095e0 tx=0x7fb270009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:17.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.814+0000 7fb26affd640 1 -- 172.21.15.64:0/4129495807 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb26c057d00 con 0x7fb27c09b0e0 2024-03-07T07:20:17.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.994+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb27c06ba40 con 0x7fb27c09b0e0 2024-03-07T07:20:17.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.994+0000 7fb26affd640 1 -- 172.21.15.64:0/4129495807 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb26c087e20 con 0x7fb27c09b0e0 2024-03-07T07:20:17.995 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-03-07T07:20:17.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb25c06b4d0 msgr2=0x7fb25c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2700095e0 tx=0x7fb270009190 comp rx=0 tx=0).stop 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c09b0e0 msgr2=0x7fb27c14e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c09b0e0 0x7fb27c14e4d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c370 tx=0x7fb26c00c830 comp rx=0 tx=0).stop 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 shutdown_connections 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb27c15f800 0x7fb27c1f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27c15fbe0 0x7fb27c0b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 --2- 172.21.15.64:0/4129495807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27c09b0e0 0x7fb27c14e4d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:17.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 >> 172.21.15.64:0/4129495807 conn(0x7fb27c0b3b90 msgr2=0x7fb27c0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 shutdown_connections 2024-03-07T07:20:17.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:17.997+0000 7fb280d67640 1 -- 172.21.15.64:0/4129495807 wait complete. 2024-03-07T07:20:18.011 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T07:20:18.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.114+0000 7f7d47d81640 1 Processor -- start 2024-03-07T07:20:18.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.114+0000 7f7d47d81640 1 -- start start 2024-03-07T07:20:18.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d47d81640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d400b5dd0 0x7f7d40144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:18.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d47d81640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d40162c80 con 0x7f7d40071410 2024-03-07T07:20:18.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d47d81640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d400b4bd0 con 0x7f7d400b43f0 2024-03-07T07:20:18.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d47d81640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d40141700 con 0x7f7d400b5dd0 2024-03-07T07:20:18.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d45b1d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f7d400b5dd0 0x7f7d40144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d4631e640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f7d40071410 0x7f7d4015e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60200/0 (socket says 172.21.15.64:60200) 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d4631e640 1 -- 172.21.15.64:0/519179739 learned_addr learned my addr 172.21.15.64:0/519179739 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d45b1d640 1 -- 172.21.15.64:0/519179739 >> v1:172.21.15.120:6789/0 conn(0x7f7d400b43f0 legacy=0x7f7d40162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d45b1d640 1 -- 172.21.15.64:0/519179739 >> v1:172.21.15.64:6789/0 conn(0x7f7d40071410 legacy=0x7f7d4015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d44b1b640 1 -- 172.21.15.64:0/519179739 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353113664 0 0) 0x7f7d400b4bd0 con 0x7f7d400b43f0 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.115+0000 7f7d45b1d640 1 -- 172.21.15.64:0/519179739 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400b4eb0 con 0x7f7d400b5dd0 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.116+0000 7f7d45b1d640 1 --2- 172.21.15.64:0/519179739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d40144540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0097a0 tx=0x7f7d3c02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ffe924185e933 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:18.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.116+0000 7f7d44b1b640 1 -- 172.21.15.64:0/519179739 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c037070 con 0x7f7d400b5dd0 2024-03-07T07:20:18.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.116+0000 7f7d44b1b640 1 -- 172.21.15.64:0/519179739 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c030710 con 0x7f7d400b5dd0 2024-03-07T07:20:18.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.116+0000 7f7d44b1b640 1 -- 172.21.15.64:0/519179739 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c030ae0 con 0x7f7d400b5dd0 2024-03-07T07:20:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.117+0000 7f7d47d81640 1 -- 172.21.15.64:0/519179739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 msgr2=0x7f7d40144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.117+0000 7f7d47d81640 1 --2- 172.21.15.64:0/519179739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d40144540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0097a0 tx=0x7f7d3c02b710 comp rx=0 tx=0).stop 2024-03-07T07:20:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.117+0000 7f7d47d81640 1 -- 172.21.15.64:0/519179739 shutdown_connections 2024-03-07T07:20:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.117+0000 7f7d47d81640 1 --2- 172.21.15.64:0/519179739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d40144540 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.117+0000 7f7d47d81640 1 -- 172.21.15.64:0/519179739 >> 172.21.15.64:0/519179739 conn(0x7f7d400d0e70 msgr2=0x7f7d400d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.117+0000 7f7d47d81640 1 -- 172.21.15.64:0/519179739 shutdown_connections 2024-03-07T07:20:18.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.117+0000 7f7d47d81640 1 -- 172.21.15.64:0/519179739 wait complete. 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.118+0000 7f7d47d81640 1 Processor -- start 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.118+0000 7f7d47d81640 1 -- start start 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d47d81640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7d40071410 0x7f7d4013e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d47d81640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d400b43f0 0x7f7d4015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d47d81640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d401f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d47d81640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40162050 con 0x7f7d400b5dd0 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d47d81640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40161e00 con 0x7f7d400b43f0 2024-03-07T07:20:18.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d47d81640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d400b61b0 con 0x7f7d40071410 2024-03-07T07:20:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d401f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:18.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d401f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60480/0 (socket says 172.21.15.64:60480) 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 -- 172.21.15.64:0/303566301 learned_addr learned my addr 172.21.15.64:0/303566301 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d45b1d640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7d40071410 0x7f7d4013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4531c640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d400b43f0 0x7f7d4015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 -- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7d40071410 msgr2=0x7f7d4013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7d40071410 0x7f7d4013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 -- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d400b43f0 msgr2=0x7f7d4015b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d400b43f0 0x7f7d4015b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 -- 172.21.15.64:0/303566301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4013eca0 con 0x7f7d400b5dd0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d4631e640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d401f5080 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800caf0 tx=0x7f7d380020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d367fc640 1 -- 172.21.15.64:0/303566301 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38018070 con 0x7f7d400b5dd0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d367fc640 1 -- 172.21.15.64:0/303566301 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800a760 con 0x7f7d400b5dd0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4013d6c0 con 0x7f7d400b5dd0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.119+0000 7f7d367fc640 1 -- 172.21.15.64:0/303566301 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800ab90 con 0x7f7d400b5dd0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.120+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4013dc80 con 0x7f7d400b5dd0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.121+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d40062c90 con 0x7f7d400b5dd0 2024-03-07T07:20:18.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.121+0000 7f7d367fc640 1 -- 172.21.15.64:0/303566301 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7d3801b070 con 0x7f7d400b5dd0 2024-03-07T07:20:18.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.121+0000 7f7d367fc640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7d2406b500 0x7f7d2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:18.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.121+0000 7f7d367fc640 1 -- 172.21.15.64:0/303566301 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d380898e0 con 0x7f7d400b5dd0 2024-03-07T07:20:18.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.122+0000 7f7d45b1d640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7d2406b500 0x7f7d2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:18.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.122+0000 7f7d45b1d640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7d2406b500 0x7f7d2406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c034000 tx=0x7f7d3c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:18.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.126+0000 7f7d367fc640 1 -- 172.21.15.64:0/303566301 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d38058110 con 0x7f7d400b5dd0 2024-03-07T07:20:18.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.323+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7d400b8a10 con 0x7f7d400b5dd0 2024-03-07T07:20:18.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.324+0000 7f7d367fc640 1 -- 172.21.15.64:0/303566301 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7d3805c1b0 con 0x7f7d400b5dd0 2024-03-07T07:20:18.324 INFO:teuthology.orchestra.run.smithi064.stderr:listed 0 entries 2024-03-07T07:20:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7d2406b500 msgr2=0x7f7d2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7d2406b500 0x7f7d2406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c034000 tx=0x7f7d3c008030 comp rx=0 tx=0).stop 2024-03-07T07:20:18.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 msgr2=0x7f7d401f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d401f5080 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800caf0 tx=0x7f7d380020f0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 shutdown_connections 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f7d2406b500 0x7f7d2406d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d400b5dd0 0x7f7d401f5080 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d400b43f0 0x7f7d4015b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 --2- 172.21.15.64:0/303566301 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7d40071410 0x7f7d4013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 >> 172.21.15.64:0/303566301 conn(0x7f7d400d0e70 msgr2=0x7f7d400b8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 shutdown_connections 2024-03-07T07:20:18.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:18.326+0000 7f7d47d81640 1 -- 172.21.15.64:0/303566301 wait complete. 2024-03-07T07:20:18.340 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-07T07:20:18.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.340 DEBUG:teuthology.orchestra.run.smithi064:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-07T07:20:18.370 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:20:18.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.371 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-07T07:20:18.425 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-07T07:20:18.426 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:20:18.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056376,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=242 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-07T07:20:18.481 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:20:18.481 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:20:18.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17807 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T07:20:18.498 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-03-07T07:20:18.498 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T07:20:18.554 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17807 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T07:20:18.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.554 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-03-07T07:20:18.569 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-07T07:20:18.569 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T07:20:18.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-07T07:20:18.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.636 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T07:20:18.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-03-07T07:20:18.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.798 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-03-07T07:20:18.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.812 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-03-07T07:20:18.882 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-03-07T07:20:18.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:20:18.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.883 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-03-07T07:20:18.899 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-07T07:20:18.899 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-07T07:20:18.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.899 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-07T07:20:18.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:18.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-07T07:20:19.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:19.035 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-03-07T07:20:19.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:19.132 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-03-07T07:20:19.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T07:20:19.146 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-03-07T07:20:19.216 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-03-07T07:20:19.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:20:19.218 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-07T07:20:19.228 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-07T07:20:19.229 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:20:19.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.331+0000 7f850a599640 1 Processor -- start 2024-03-07T07:20:19.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.331+0000 7f850a599640 1 -- start start 2024-03-07T07:20:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f850a599640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8504144540 0x7f8504162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f850a599640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85040b4bd0 con 0x7f85040b5dd0 2024-03-07T07:20:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f850a599640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85040b4eb0 con 0x7f8504162ab0 2024-03-07T07:20:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f850a599640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8504141700 con 0x7f8504144540 2024-03-07T07:20:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f8503fff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f85040b5dd0 0x7f85040b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60206/0 (socket says 172.21.15.64:60206) 2024-03-07T07:20:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f8503fff640 1 -- 172.21.15.64:0/578163064 learned_addr learned my addr 172.21.15.64:0/578163064 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741060009 0 0) 0x7f85040b4bd0 con 0x7f85040b5dd0 2024-03-07T07:20:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f0003520 con 0x7f85040b5dd0 2024-03-07T07:20:19.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.332+0000 7f85037fe640 1 --2- 172.21.15.64:0/578163064 >> v2:172.21.15.120:3300/0 conn(0x7f8504144540 0x7f8504162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2458595698 0 0) 0x7f84f0003520 con 0x7f85040b5dd0 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 >> v2:172.21.15.120:3300/0 conn(0x7f8504144540 msgr2=0x7f8504162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 --2- 172.21.15.64:0/578163064 >> v2:172.21.15.120:3300/0 conn(0x7f8504144540 0x7f8504162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 >> v1:172.21.15.120:6789/0 conn(0x7f8504162ab0 legacy=0x7f8504071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040b5190 con 0x7f85040b5dd0 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f84f4001670 con 0x7f85040b5dd0 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 >> v1:172.21.15.64:6789/0 conn(0x7f85040b5dd0 legacy=0x7f85040b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84f00039b0 0x7f84f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00043b0 0x7f84f0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f0005120 0x7f84f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8504141700 con 0x7f84f0005120 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f000c060 con 0x7f84f00039b0 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f000c260 con 0x7f84f00043b0 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8508b36640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f0005120 0x7f84f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8503fff640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84f00039b0 0x7f84f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f85037fe640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00043b0 0x7f84f0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8503fff640 1 -- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00043b0 msgr2=0x7f84f0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8503fff640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00043b0 0x7f84f0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8503fff640 1 -- 172.21.15.64:0/578163064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f0005120 msgr2=0x7f84f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8503fff640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f0005120 0x7f84f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8503fff640 1 -- 172.21.15.64:0/578163064 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0003520 con 0x7f84f00039b0 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8508b36640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84f0005120 0x7f84f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.333+0000 7f8508b36640 1 -- 172.21.15.64:0/578163064 reap_dead start 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.334+0000 7f8503fff640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84f00039b0 0x7f84f0003d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c240 tx=0x7f84f0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25d70691f8ba19f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.334+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4007d10 con 0x7f84f00039b0 2024-03-07T07:20:19.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.334+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4030070 con 0x7f84f00039b0 2024-03-07T07:20:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.334+0000 7f8502ffd640 1 -- 172.21.15.64:0/578163064 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f402cda0 con 0x7f84f00039b0 2024-03-07T07:20:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.335+0000 7f850a599640 1 -- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84f00039b0 msgr2=0x7f84f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.335+0000 7f850a599640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84f00039b0 0x7f84f0003d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c240 tx=0x7f84f0005830 comp rx=0 tx=0).stop 2024-03-07T07:20:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.335+0000 7f850a599640 1 -- 172.21.15.64:0/578163064 shutdown_connections 2024-03-07T07:20:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.335+0000 7f850a599640 1 --2- 172.21.15.64:0/578163064 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84f00039b0 0x7f84f0003d90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.335+0000 7f850a599640 1 -- 172.21.15.64:0/578163064 >> 172.21.15.64:0/578163064 conn(0x7f85040d0e70 msgr2=0x7f85040d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:19.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.335+0000 7f850a599640 1 -- 172.21.15.64:0/578163064 shutdown_connections 2024-03-07T07:20:19.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.335+0000 7f850a599640 1 -- 172.21.15.64:0/578163064 wait complete. 2024-03-07T07:20:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.336+0000 7f850a599640 1 Processor -- start 2024-03-07T07:20:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.336+0000 7f850a599640 1 -- start start 2024-03-07T07:20:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f850a599640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00039b0 0x7f8504070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f850a599640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504070c90 0x7f85041f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f850a599640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85041f4290 0x7f850415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f850a599640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85040b61b0 con 0x7f85041f4290 2024-03-07T07:20:19.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f850a599640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8504161e00 con 0x7f8504070c90 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f850a599640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8504162570 con 0x7f84f00039b0 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85041f4290 0x7f850415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85041f4290 0x7f850415acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60502/0 (socket says 172.21.15.64:60502) 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 -- 172.21.15.64:0/3123600270 learned_addr learned my addr 172.21.15.64:0/3123600270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8503fff640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00039b0 0x7f8504070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f85037fe640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504070c90 0x7f85041f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 -- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00039b0 msgr2=0x7f8504070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00039b0 0x7f8504070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 -- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504070c90 msgr2=0x7f85041f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504070c90 0x7f85041f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 -- 172.21.15.64:0/3123600270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85041f4710 con 0x7f85041f4290 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8508b36640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85041f4290 0x7f850415acd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f85040d20e0 tx=0x7f84fc00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.337+0000 7f8500ff9640 1 -- 172.21.15.64:0/3123600270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc001b30 con 0x7f85041f4290 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.338+0000 7f8500ff9640 1 -- 172.21.15.64:0/3123600270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc003b80 con 0x7f85041f4290 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.338+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8504141a50 con 0x7f85041f4290 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.338+0000 7f8500ff9640 1 -- 172.21.15.64:0/3123600270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc003d70 con 0x7f85041f4290 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.338+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850415b940 con 0x7f85041f4290 2024-03-07T07:20:19.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.339+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8504062c90 con 0x7f85041f4290 2024-03-07T07:20:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.339+0000 7f8500ff9640 1 -- 172.21.15.64:0/3123600270 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f84fc01b020 con 0x7f85041f4290 2024-03-07T07:20:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.339+0000 7f8500ff9640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f84e406b4d0 0x7f84e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.340+0000 7f8500ff9640 1 -- 172.21.15.64:0/3123600270 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84fc089920 con 0x7f85041f4290 2024-03-07T07:20:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.340+0000 7f8503fff640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f84e406b4d0 0x7f84e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.340+0000 7f8503fff640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f84e406b4d0 0x7f84e406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f4007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:19.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.348+0000 7f8500ff9640 1 -- 172.21.15.64:0/3123600270 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f84fc0581e0 con 0x7f85041f4290 2024-03-07T07:20:19.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.526+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8504071070 con 0x7f84e406b4d0 2024-03-07T07:20:19.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.532+0000 7f8500ff9640 1 -- 172.21.15.64:0/3123600270 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251876 (secure 0 0 0) 0x7f84f4083800 con 0x7f84e406b4d0 2024-03-07T07:20:19.535 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:19.536 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:20:19.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f84e406b4d0 msgr2=0x7f84e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f84e406b4d0 0x7f84e406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f4007000 comp rx=0 tx=0).stop 2024-03-07T07:20:19.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85041f4290 msgr2=0x7f850415acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85041f4290 0x7f850415acd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f85040d20e0 tx=0x7f84fc00feb0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 shutdown_connections 2024-03-07T07:20:19.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f84e406b4d0 0x7f84e406d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85041f4290 0x7f850415acd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8504070c90 0x7f85041f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 --2- 172.21.15.64:0/3123600270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84f00039b0 0x7f8504070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 >> 172.21.15.64:0/3123600270 conn(0x7f85040d0e70 msgr2=0x7f850415e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:19.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 shutdown_connections 2024-03-07T07:20:19.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.535+0000 7f850a599640 1 -- 172.21.15.64:0/3123600270 wait complete. 2024-03-07T07:20:19.546 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-03-07T07:20:18.191640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593034,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002119"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018002+0000","last_change":"2024-03-07T07:20:08.025425+0000","last_active":"2024-03-07T07:20:09.018002+0000","last_peered":"2024-03-07T07:20:09.018002+0000","last_clean":"2024-03-07T07:20:09.018002+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:09.018002+0000","last_undegraded":"2024-03-07T07:20:09.018002+0000","last_fullsized":"2024-03-07T07:20:09.018002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516922+0000","last_change":"2024-03-07T07:20:08.036714+0000","last_active":"2024-03-07T07:20:09.516922+0000","last_peered":"2024-03-07T07:20:09.516922+0000","last_clean":"2024-03-07T07:20:09.516922+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:09.516922+0000","last_undegraded":"2024-03-07T07:20:09.516922+0000","last_fullsized":"2024-03-07T07:20:09.516922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:43.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576955+0000","last_change":"2024-03-07T07:20:08.037973+0000","last_active":"2024-03-07T07:20:09.576955+0000","last_peered":"2024-03-07T07:20:09.576955+0000","last_clean":"2024-03-07T07:20:09.576955+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:09.576955+0000","last_undegraded":"2024-03-07T07:20:09.576955+0000","last_fullsized":"2024-03-07T07:20:09.576955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575962+0000","last_change":"2024-03-07T07:20:08.042993+0000","last_active":"2024-03-07T07:20:09.575962+0000","last_peered":"2024-03-07T07:20:09.575962+0000","last_clean":"2024-03-07T07:20:09.575962+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:09.575962+0000","last_undegraded":"2024-03-07T07:20:09.575962+0000","last_fullsized":"2024-03-07T07:20:09.575962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575521+0000","last_change":"2024-03-07T07:20:08.037872+0000","last_active":"2024-03-07T07:20:09.575521+0000","last_peered":"2024-03-07T07:20:09.575521+0000","last_clean":"2024-03-07T07:20:09.575521+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:09.575521+0000","last_undegraded":"2024-03-07T07:20:09.575521+0000","last_fullsized":"2024-03-07T07:20:09.575521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:47.883124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516617+0000","last_change":"2024-03-07T07:20:08.028212+0000","last_active":"2024-03-07T07:20:09.516617+0000","last_peered":"2024-03-07T07:20:09.516617+0000","last_clean":"2024-03-07T07:20:09.516617+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:09.516617+0000","last_undegraded":"2024-03-07T07:20:09.516617+0000","last_fullsized":"2024-03-07T07:20:09.516617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577021+0000","last_change":"2024-03-07T07:20:08.032379+0000","last_active":"2024-03-07T07:20:09.577021+0000","last_peered":"2024-03-07T07:20:09.577021+0000","last_clean":"2024-03-07T07:20:09.577021+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:09.577021+0000","last_undegraded":"2024-03-07T07:20:09.577021+0000","last_fullsized":"2024-03-07T07:20:09.577021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576904+0000","last_change":"2024-03-07T07:20:08.035495+0000","last_active":"2024-03-07T07:20:09.576904+0000","last_peered":"2024-03-07T07:20:09.576904+0000","last_clean":"2024-03-07T07:20:09.576904+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:09.576904+0000","last_undegraded":"2024-03-07T07:20:09.576904+0000","last_fullsized":"2024-03-07T07:20:09.576904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018731+0000","last_change":"2024-03-07T07:20:08.034012+0000","last_active":"2024-03-07T07:20:09.018731+0000","last_peered":"2024-03-07T07:20:09.018731+0000","last_clean":"2024-03-07T07:20:09.018731+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:09.018731+0000","last_undegraded":"2024-03-07T07:20:09.018731+0000","last_fullsized":"2024-03-07T07:20:09.018731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576070+0000","last_change":"2024-03-07T07:20:08.037279+0000","last_active":"2024-03-07T07:20:09.576070+0000","last_peered":"2024-03-07T07:20:09.576070+0000","last_clean":"2024-03-07T07:20:09.576070+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:09.576070+0000","last_undegraded":"2024-03-07T07:20:09.576070+0000","last_fullsized":"2024-03-07T07:20:09.576070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576985+0000","last_change":"2024-03-07T07:20:08.039682+0000","last_active":"2024-03-07T07:20:09.576985+0000","last_peered":"2024-03-07T07:20:09.576985+0000","last_clean":"2024-03-07T07:20:09.576985+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:09.576985+0000","last_undegraded":"2024-03-07T07:20:09.576985+0000","last_fullsized":"2024-03-07T07:20:09.576985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576401+0000","last_change":"2024-03-07T07:20:08.028305+0000","last_active":"2024-03-07T07:20:09.576401+0000","last_peered":"2024-03-07T07:20:09.576401+0000","last_clean":"2024-03-07T07:20:09.576401+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:09.576401+0000","last_undegraded":"2024-03-07T07:20:09.576401+0000","last_fullsized":"2024-03-07T07:20:09.576401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575664+0000","last_change":"2024-03-07T07:20:08.033602+0000","last_active":"2024-03-07T07:20:09.575664+0000","last_peered":"2024-03-07T07:20:09.575664+0000","last_clean":"2024-03-07T07:20:09.575664+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:09.575664+0000","last_undegraded":"2024-03-07T07:20:09.575664+0000","last_fullsized":"2024-03-07T07:20:09.575664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:55.373070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516717+0000","last_change":"2024-03-07T07:20:08.022351+0000","last_active":"2024-03-07T07:20:09.516717+0000","last_peered":"2024-03-07T07:20:09.516717+0000","last_clean":"2024-03-07T07:20:09.516717+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:09.516717+0000","last_undegraded":"2024-03-07T07:20:09.516717+0000","last_fullsized":"2024-03-07T07:20:09.516717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:20:08.021148+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577186+0000","last_change":"2024-03-07T07:20:08.035791+0000","last_active":"2024-03-07T07:20:09.577186+0000","last_peered":"2024-03-07T07:20:09.577186+0000","last_clean":"2024-03-07T07:20:09.577186+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:09.577186+0000","last_undegraded":"2024-03-07T07:20:09.577186+0000","last_fullsized":"2024-03-07T07:20:09.577186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576963+0000","last_change":"2024-03-07T07:20:08.034016+0000","last_active":"2024-03-07T07:20:09.576963+0000","last_peered":"2024-03-07T07:20:09.576963+0000","last_clean":"2024-03-07T07:20:09.576963+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:09.576963+0000","last_undegraded":"2024-03-07T07:20:09.576963+0000","last_fullsized":"2024-03-07T07:20:09.576963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517000+0000","last_change":"2024-03-07T07:20:08.035489+0000","last_active":"2024-03-07T07:20:09.517000+0000","last_peered":"2024-03-07T07:20:09.517000+0000","last_clean":"2024-03-07T07:20:09.517000+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:09.517000+0000","last_undegraded":"2024-03-07T07:20:09.517000+0000","last_fullsized":"2024-03-07T07:20:09.517000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:07.455855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516748+0000","last_change":"2024-03-07T07:20:08.037898+0000","last_active":"2024-03-07T07:20:09.516748+0000","last_peered":"2024-03-07T07:20:09.516748+0000","last_clean":"2024-03-07T07:20:09.516748+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:09.516748+0000","last_undegraded":"2024-03-07T07:20:09.516748+0000","last_fullsized":"2024-03-07T07:20:09.516748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575696+0000","last_change":"2024-03-07T07:20:08.025458+0000","last_active":"2024-03-07T07:20:09.575696+0000","last_peered":"2024-03-07T07:20:09.575696+0000","last_clean":"2024-03-07T07:20:09.575696+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:09.575696+0000","last_undegraded":"2024-03-07T07:20:09.575696+0000","last_fullsized":"2024-03-07T07:20:09.575696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:24.155245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018155+0000","last_change":"2024-03-07T07:20:08.040577+0000","last_active":"2024-03-07T07:20:09.018155+0000","last_peered":"2024-03-07T07:20:09.018155+0000","last_clean":"2024-03-07T07:20:09.018155+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:09.018155+0000","last_undegraded":"2024-03-07T07:20:09.018155+0000","last_fullsized":"2024-03-07T07:20:09.018155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018038+0000","last_change":"2024-03-07T07:20:08.039803+0000","last_active":"2024-03-07T07:20:09.018038+0000","last_peered":"2024-03-07T07:20:09.018038+0000","last_clean":"2024-03-07T07:20:09.018038+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:09.018038+0000","last_undegraded":"2024-03-07T07:20:09.018038+0000","last_fullsized":"2024-03-07T07:20:09.018038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015233+0000","last_change":"2024-03-07T07:20:08.031321+0000","last_active":"2024-03-07T07:20:09.015233+0000","last_peered":"2024-03-07T07:20:09.015233+0000","last_clean":"2024-03-07T07:20:09.015233+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:09.015233+0000","last_undegraded":"2024-03-07T07:20:09.015233+0000","last_fullsized":"2024-03-07T07:20:09.015233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576999+0000","last_change":"2024-03-07T07:20:08.024172+0000","last_active":"2024-03-07T07:20:09.576999+0000","last_peered":"2024-03-07T07:20:09.576999+0000","last_clean":"2024-03-07T07:20:09.576999+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:09.576999+0000","last_undegraded":"2024-03-07T07:20:09.576999+0000","last_fullsized":"2024-03-07T07:20:09.576999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576283+0000","last_change":"2024-03-07T07:20:08.033067+0000","last_active":"2024-03-07T07:20:09.576283+0000","last_peered":"2024-03-07T07:20:09.576283+0000","last_clean":"2024-03-07T07:20:09.576283+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:09.576283+0000","last_undegraded":"2024-03-07T07:20:09.576283+0000","last_fullsized":"2024-03-07T07:20:09.576283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517079+0000","last_change":"2024-03-07T07:20:08.031519+0000","last_active":"2024-03-07T07:20:09.517079+0000","last_peered":"2024-03-07T07:20:09.517079+0000","last_clean":"2024-03-07T07:20:09.517079+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:09.517079+0000","last_undegraded":"2024-03-07T07:20:09.517079+0000","last_fullsized":"2024-03-07T07:20:09.517079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:33.954019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576399+0000","last_change":"2024-03-07T07:20:08.043585+0000","last_active":"2024-03-07T07:20:09.576399+0000","last_peered":"2024-03-07T07:20:09.576399+0000","last_clean":"2024-03-07T07:20:09.576399+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:09.576399+0000","last_undegraded":"2024-03-07T07:20:09.576399+0000","last_fullsized":"2024-03-07T07:20:09.576399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576561+0000","last_change":"2024-03-07T07:20:08.021978+0000","last_active":"2024-03-07T07:20:09.576561+0000","last_peered":"2024-03-07T07:20:09.576561+0000","last_clean":"2024-03-07T07:20:09.576561+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:09.576561+0000","last_undegraded":"2024-03-07T07:20:09.576561+0000","last_fullsized":"2024-03-07T07:20:09.576561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018178+0000","last_change":"2024-03-07T07:20:08.040950+0000","last_active":"2024-03-07T07:20:09.018178+0000","last_peered":"2024-03-07T07:20:09.018178+0000","last_clean":"2024-03-07T07:20:09.018178+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:09.018178+0000","last_undegraded":"2024-03-07T07:20:09.018178+0000","last_fullsized":"2024-03-07T07:20:09.018178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576528+0000","last_change":"2024-03-07T07:20:08.031636+0000","last_active":"2024-03-07T07:20:09.576528+0000","last_peered":"2024-03-07T07:20:09.576528+0000","last_clean":"2024-03-07T07:20:09.576528+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:09.576528+0000","last_undegraded":"2024-03-07T07:20:09.576528+0000","last_fullsized":"2024-03-07T07:20:09.576528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017643+0000","last_change":"2024-03-07T07:20:08.023430+0000","last_active":"2024-03-07T07:20:09.017643+0000","last_peered":"2024-03-07T07:20:09.017643+0000","last_clean":"2024-03-07T07:20:09.017643+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:09.017643+0000","last_undegraded":"2024-03-07T07:20:09.017643+0000","last_fullsized":"2024-03-07T07:20:09.017643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:29.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576960+0000","last_change":"2024-03-07T07:20:08.033808+0000","last_active":"2024-03-07T07:20:09.576960+0000","last_peered":"2024-03-07T07:20:09.576960+0000","last_clean":"2024-03-07T07:20:09.576960+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:09.576960+0000","last_undegraded":"2024-03-07T07:20:09.576960+0000","last_fullsized":"2024-03-07T07:20:09.576960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015411+0000","last_change":"2024-03-07T07:20:08.030548+0000","last_active":"2024-03-07T07:20:09.015411+0000","last_peered":"2024-03-07T07:20:09.015411+0000","last_clean":"2024-03-07T07:20:09.015411+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:09.015411+0000","last_undegraded":"2024-03-07T07:20:09.015411+0000","last_fullsized":"2024-03-07T07:20:09.015411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018115+0000","last_change":"2024-03-07T07:20:07.025352+0000","last_active":"2024-03-07T07:20:09.018115+0000","last_peered":"2024-03-07T07:20:09.018115+0000","last_clean":"2024-03-07T07:20:09.018115+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:09.018115+0000","last_undegraded":"2024-03-07T07:20:09.018115+0000","last_fullsized":"2024-03-07T07:20:09.018115+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018004+0000","last_change":"2024-03-07T07:20:08.033169+0000","last_active":"2024-03-07T07:20:09.018004+0000","last_peered":"2024-03-07T07:20:09.018004+0000","last_clean":"2024-03-07T07:20:09.018004+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:09.018004+0000","last_undegraded":"2024-03-07T07:20:09.018004+0000","last_fullsized":"2024-03-07T07:20:09.018004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015349+0000","last_change":"2024-03-07T07:20:07.025175+0000","last_active":"2024-03-07T07:20:09.015349+0000","last_peered":"2024-03-07T07:20:09.015349+0000","last_clean":"2024-03-07T07:20:09.015349+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:09.015349+0000","last_undegraded":"2024-03-07T07:20:09.015349+0000","last_fullsized":"2024-03-07T07:20:09.015349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:23.151665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018559+0000","last_change":"2024-03-07T07:20:08.040222+0000","last_active":"2024-03-07T07:20:09.018559+0000","last_peered":"2024-03-07T07:20:09.018559+0000","last_clean":"2024-03-07T07:20:09.018559+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:09.018559+0000","last_undegraded":"2024-03-07T07:20:09.018559+0000","last_fullsized":"2024-03-07T07:20:09.018559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516719+0000","last_change":"2024-03-07T07:20:07.033158+0000","last_active":"2024-03-07T07:20:09.516719+0000","last_peered":"2024-03-07T07:20:09.516719+0000","last_clean":"2024-03-07T07:20:09.516719+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:09.516719+0000","last_undegraded":"2024-03-07T07:20:09.516719+0000","last_fullsized":"2024-03-07T07:20:09.516719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:30.672111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576436+0000","last_change":"2024-03-07T07:20:08.043374+0000","last_active":"2024-03-07T07:20:09.576436+0000","last_peered":"2024-03-07T07:20:09.576436+0000","last_clean":"2024-03-07T07:20:09.576436+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:09.576436+0000","last_undegraded":"2024-03-07T07:20:09.576436+0000","last_fullsized":"2024-03-07T07:20:09.576436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575854+0000","last_change":"2024-03-07T07:20:07.028373+0000","last_active":"2024-03-07T07:20:09.575854+0000","last_peered":"2024-03-07T07:20:09.575854+0000","last_clean":"2024-03-07T07:20:09.575854+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:09.575854+0000","last_undegraded":"2024-03-07T07:20:09.575854+0000","last_fullsized":"2024-03-07T07:20:09.575854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:51.229977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576660+0000","last_change":"2024-03-07T07:20:08.033882+0000","last_active":"2024-03-07T07:20:09.576660+0000","last_peered":"2024-03-07T07:20:09.576660+0000","last_clean":"2024-03-07T07:20:09.576660+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:09.576660+0000","last_undegraded":"2024-03-07T07:20:09.576660+0000","last_fullsized":"2024-03-07T07:20:09.576660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018255+0000","last_change":"2024-03-07T07:20:07.028711+0000","last_active":"2024-03-07T07:20:09.018255+0000","last_peered":"2024-03-07T07:20:09.018255+0000","last_clean":"2024-03-07T07:20:09.018255+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:09.018255+0000","last_undegraded":"2024-03-07T07:20:09.018255+0000","last_fullsized":"2024-03-07T07:20:09.018255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:46.879544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576378+0000","last_change":"2024-03-07T07:20:08.030098+0000","last_active":"2024-03-07T07:20:09.576378+0000","last_peered":"2024-03-07T07:20:09.576378+0000","last_clean":"2024-03-07T07:20:09.576378+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:09.576378+0000","last_undegraded":"2024-03-07T07:20:09.576378+0000","last_fullsized":"2024-03-07T07:20:09.576378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018003+0000","last_change":"2024-03-07T07:20:07.029834+0000","last_active":"2024-03-07T07:20:09.018003+0000","last_peered":"2024-03-07T07:20:09.018003+0000","last_clean":"2024-03-07T07:20:09.018003+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:09.018003+0000","last_undegraded":"2024-03-07T07:20:09.018003+0000","last_fullsized":"2024-03-07T07:20:09.018003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:55:49.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018248+0000","last_change":"2024-03-07T07:20:08.028053+0000","last_active":"2024-03-07T07:20:09.018248+0000","last_peered":"2024-03-07T07:20:09.018248+0000","last_clean":"2024-03-07T07:20:09.018248+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:09.018248+0000","last_undegraded":"2024-03-07T07:20:09.018248+0000","last_fullsized":"2024-03-07T07:20:09.018248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516477+0000","last_change":"2024-03-07T07:20:07.033411+0000","last_active":"2024-03-07T07:20:09.516477+0000","last_peered":"2024-03-07T07:20:09.516477+0000","last_clean":"2024-03-07T07:20:09.516477+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:09.516477+0000","last_undegraded":"2024-03-07T07:20:09.516477+0000","last_fullsized":"2024-03-07T07:20:09.516477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:15.200144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576038+0000","last_change":"2024-03-07T07:20:08.032643+0000","last_active":"2024-03-07T07:20:09.576038+0000","last_peered":"2024-03-07T07:20:09.576038+0000","last_clean":"2024-03-07T07:20:09.576038+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:09.576038+0000","last_undegraded":"2024-03-07T07:20:09.576038+0000","last_fullsized":"2024-03-07T07:20:09.576038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018364+0000","last_change":"2024-03-07T07:20:07.027022+0000","last_active":"2024-03-07T07:20:09.018364+0000","last_peered":"2024-03-07T07:20:09.018364+0000","last_clean":"2024-03-07T07:20:09.018364+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:09.018364+0000","last_undegraded":"2024-03-07T07:20:09.018364+0000","last_fullsized":"2024-03-07T07:20:09.018364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:35.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.028592+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516961+0000","last_change":"2024-03-07T07:20:07.030367+0000","last_active":"2024-03-07T07:20:09.516961+0000","last_peered":"2024-03-07T07:20:09.516961+0000","last_clean":"2024-03-07T07:20:09.516961+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:09.516961+0000","last_undegraded":"2024-03-07T07:20:09.516961+0000","last_fullsized":"2024-03-07T07:20:09.516961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:12.128114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516860+0000","last_change":"2024-03-07T07:20:08.034949+0000","last_active":"2024-03-07T07:20:09.516860+0000","last_peered":"2024-03-07T07:20:09.516860+0000","last_clean":"2024-03-07T07:20:09.516860+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:09.516860+0000","last_undegraded":"2024-03-07T07:20:09.516860+0000","last_fullsized":"2024-03-07T07:20:09.516860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015409+0000","last_change":"2024-03-07T07:20:07.030049+0000","last_active":"2024-03-07T07:20:09.015409+0000","last_peered":"2024-03-07T07:20:09.015409+0000","last_clean":"2024-03-07T07:20:09.015409+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:09.015409+0000","last_undegraded":"2024-03-07T07:20:09.015409+0000","last_fullsized":"2024-03-07T07:20:09.015409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516872+0000","last_change":"2024-03-07T07:20:08.030884+0000","last_active":"2024-03-07T07:20:09.516872+0000","last_peered":"2024-03-07T07:20:09.516872+0000","last_clean":"2024-03-07T07:20:09.516872+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:09.516872+0000","last_undegraded":"2024-03-07T07:20:09.516872+0000","last_fullsized":"2024-03-07T07:20:09.516872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:22.617306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577049+0000","last_change":"2024-03-07T07:20:07.028526+0000","last_active":"2024-03-07T07:20:09.577049+0000","last_peered":"2024-03-07T07:20:09.577049+0000","last_clean":"2024-03-07T07:20:09.577049+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:09.577049+0000","last_undegraded":"2024-03-07T07:20:09.577049+0000","last_fullsized":"2024-03-07T07:20:09.577049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576521+0000","last_change":"2024-03-07T07:20:08.038594+0000","last_active":"2024-03-07T07:20:09.576521+0000","last_peered":"2024-03-07T07:20:09.576521+0000","last_clean":"2024-03-07T07:20:09.576521+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:09.576521+0000","last_undegraded":"2024-03-07T07:20:09.576521+0000","last_fullsized":"2024-03-07T07:20:09.576521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'7","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516823+0000","last_change":"2024-03-07T07:20:07.035824+0000","last_active":"2024-03-07T07:20:09.516823+0000","last_peered":"2024-03-07T07:20:09.516823+0000","last_clean":"2024-03-07T07:20:09.516823+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:09.516823+0000","last_undegraded":"2024-03-07T07:20:09.516823+0000","last_fullsized":"2024-03-07T07:20:09.516823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576168+0000","last_change":"2024-03-07T07:20:08.034611+0000","last_active":"2024-03-07T07:20:09.576168+0000","last_peered":"2024-03-07T07:20:09.576168+0000","last_clean":"2024-03-07T07:20:09.576168+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:09.576168+0000","last_undegraded":"2024-03-07T07:20:09.576168+0000","last_fullsized":"2024-03-07T07:20:09.576168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018274+0000","last_change":"2024-03-07T07:20:07.025705+0000","last_active":"2024-03-07T07:20:09.018274+0000","last_peered":"2024-03-07T07:20:09.018274+0000","last_clean":"2024-03-07T07:20:09.018274+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:09.018274+0000","last_undegraded":"2024-03-07T07:20:09.018274+0000","last_fullsized":"2024-03-07T07:20:09.018274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516759+0000","last_change":"2024-03-07T07:20:08.037145+0000","last_active":"2024-03-07T07:20:09.516759+0000","last_peered":"2024-03-07T07:20:09.516759+0000","last_clean":"2024-03-07T07:20:09.516759+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:09.516759+0000","last_undegraded":"2024-03-07T07:20:09.516759+0000","last_fullsized":"2024-03-07T07:20:09.516759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576262+0000","last_change":"2024-03-07T07:20:07.026780+0000","last_active":"2024-03-07T07:20:09.576262+0000","last_peered":"2024-03-07T07:20:09.576262+0000","last_clean":"2024-03-07T07:20:09.576262+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:09.576262+0000","last_undegraded":"2024-03-07T07:20:09.576262+0000","last_fullsized":"2024-03-07T07:20:09.576262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577126+0000","last_change":"2024-03-07T07:20:08.033869+0000","last_active":"2024-03-07T07:20:09.577126+0000","last_peered":"2024-03-07T07:20:09.577126+0000","last_clean":"2024-03-07T07:20:09.577126+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:09.577126+0000","last_undegraded":"2024-03-07T07:20:09.577126+0000","last_fullsized":"2024-03-07T07:20:09.577126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516453+0000","last_change":"2024-03-07T07:20:07.032636+0000","last_active":"2024-03-07T07:20:09.516453+0000","last_peered":"2024-03-07T07:20:09.516453+0000","last_clean":"2024-03-07T07:20:09.516453+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:09.516453+0000","last_undegraded":"2024-03-07T07:20:09.516453+0000","last_fullsized":"2024-03-07T07:20:09.516453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017781+0000","last_change":"2024-03-07T07:20:08.036782+0000","last_active":"2024-03-07T07:20:09.017781+0000","last_peered":"2024-03-07T07:20:09.017781+0000","last_clean":"2024-03-07T07:20:09.017781+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:09.017781+0000","last_undegraded":"2024-03-07T07:20:09.017781+0000","last_fullsized":"2024-03-07T07:20:09.017781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:19.197402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516567+0000","last_change":"2024-03-07T07:20:07.033230+0000","last_active":"2024-03-07T07:20:09.516567+0000","last_peered":"2024-03-07T07:20:09.516567+0000","last_clean":"2024-03-07T07:20:09.516567+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:09.516567+0000","last_undegraded":"2024-03-07T07:20:09.516567+0000","last_fullsized":"2024-03-07T07:20:09.516567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:42.777795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018625+0000","last_change":"2024-03-07T07:20:08.036934+0000","last_active":"2024-03-07T07:20:09.018625+0000","last_peered":"2024-03-07T07:20:09.018625+0000","last_clean":"2024-03-07T07:20:09.018625+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:09.018625+0000","last_undegraded":"2024-03-07T07:20:09.018625+0000","last_fullsized":"2024-03-07T07:20:09.018625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576460+0000","last_change":"2024-03-07T07:20:07.033018+0000","last_active":"2024-03-07T07:20:09.576460+0000","last_peered":"2024-03-07T07:20:09.576460+0000","last_clean":"2024-03-07T07:20:09.576460+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:09.576460+0000","last_undegraded":"2024-03-07T07:20:09.576460+0000","last_fullsized":"2024-03-07T07:20:09.576460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577155+0000","last_change":"2024-03-07T07:20:08.035720+0000","last_active":"2024-03-07T07:20:09.577155+0000","last_peered":"2024-03-07T07:20:09.577155+0000","last_clean":"2024-03-07T07:20:09.577155+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:09.577155+0000","last_undegraded":"2024-03-07T07:20:09.577155+0000","last_fullsized":"2024-03-07T07:20:09.577155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018650+0000","last_change":"2024-03-07T07:20:07.026560+0000","last_active":"2024-03-07T07:20:09.018650+0000","last_peered":"2024-03-07T07:20:09.018650+0000","last_clean":"2024-03-07T07:20:09.018650+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:09.018650+0000","last_undegraded":"2024-03-07T07:20:09.018650+0000","last_fullsized":"2024-03-07T07:20:09.018650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:52.614111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576186+0000","last_change":"2024-03-07T07:20:08.033852+0000","last_active":"2024-03-07T07:20:09.576186+0000","last_peered":"2024-03-07T07:20:09.576186+0000","last_clean":"2024-03-07T07:20:09.576186+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:09.576186+0000","last_undegraded":"2024-03-07T07:20:09.576186+0000","last_fullsized":"2024-03-07T07:20:09.576186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018510+0000","last_change":"2024-03-07T07:20:07.023668+0000","last_active":"2024-03-07T07:20:09.018510+0000","last_peered":"2024-03-07T07:20:09.018510+0000","last_clean":"2024-03-07T07:20:09.018510+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:09.018510+0000","last_undegraded":"2024-03-07T07:20:09.018510+0000","last_fullsized":"2024-03-07T07:20:09.018510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:07.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575815+0000","last_change":"2024-03-07T07:20:08.030524+0000","last_active":"2024-03-07T07:20:09.575815+0000","last_peered":"2024-03-07T07:20:09.575815+0000","last_clean":"2024-03-07T07:20:09.575815+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:09.575815+0000","last_undegraded":"2024-03-07T07:20:09.575815+0000","last_fullsized":"2024-03-07T07:20:09.575815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:18.342434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018550+0000","last_change":"2024-03-07T07:20:07.029892+0000","last_active":"2024-03-07T07:20:09.018550+0000","last_peered":"2024-03-07T07:20:09.018550+0000","last_clean":"2024-03-07T07:20:09.018550+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:09.018550+0000","last_undegraded":"2024-03-07T07:20:09.018550+0000","last_fullsized":"2024-03-07T07:20:09.018550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:24.735113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575612+0000","last_change":"2024-03-07T07:20:08.032134+0000","last_active":"2024-03-07T07:20:09.575612+0000","last_peered":"2024-03-07T07:20:09.575612+0000","last_clean":"2024-03-07T07:20:09.575612+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:09.575612+0000","last_undegraded":"2024-03-07T07:20:09.575612+0000","last_fullsized":"2024-03-07T07:20:09.575612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:48.726782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015375+0000","last_change":"2024-03-07T07:20:07.028153+0000","last_active":"2024-03-07T07:20:09.015375+0000","last_peered":"2024-03-07T07:20:09.015375+0000","last_clean":"2024-03-07T07:20:09.015375+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:09.015375+0000","last_undegraded":"2024-03-07T07:20:09.015375+0000","last_fullsized":"2024-03-07T07:20:09.015375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575806+0000","last_change":"2024-03-07T07:20:08.032594+0000","last_active":"2024-03-07T07:20:09.575806+0000","last_peered":"2024-03-07T07:20:09.575806+0000","last_clean":"2024-03-07T07:20:09.575806+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:09.575806+0000","last_undegraded":"2024-03-07T07:20:09.575806+0000","last_fullsized":"2024-03-07T07:20:09.575806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:15.552375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018303+0000","last_change":"2024-03-07T07:20:07.028455+0000","last_active":"2024-03-07T07:20:09.018303+0000","last_peered":"2024-03-07T07:20:09.018303+0000","last_clean":"2024-03-07T07:20:09.018303+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:09.018303+0000","last_undegraded":"2024-03-07T07:20:09.018303+0000","last_fullsized":"2024-03-07T07:20:09.018303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575541+0000","last_change":"2024-03-07T07:20:08.031299+0000","last_active":"2024-03-07T07:20:09.575541+0000","last_peered":"2024-03-07T07:20:09.575541+0000","last_clean":"2024-03-07T07:20:09.575541+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:09.575541+0000","last_undegraded":"2024-03-07T07:20:09.575541+0000","last_fullsized":"2024-03-07T07:20:09.575541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:24:00.520723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017852+0000","last_change":"2024-03-07T07:20:07.028369+0000","last_active":"2024-03-07T07:20:09.017852+0000","last_peered":"2024-03-07T07:20:09.017852+0000","last_clean":"2024-03-07T07:20:09.017852+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:09.017852+0000","last_undegraded":"2024-03-07T07:20:09.017852+0000","last_fullsized":"2024-03-07T07:20:09.017852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.033176+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576122+0000","last_change":"2024-03-07T07:20:07.029244+0000","last_active":"2024-03-07T07:20:09.576122+0000","last_peered":"2024-03-07T07:20:09.576122+0000","last_clean":"2024-03-07T07:20:09.576122+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:09.576122+0000","last_undegraded":"2024-03-07T07:20:09.576122+0000","last_fullsized":"2024-03-07T07:20:09.576122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575628+0000","last_change":"2024-03-07T07:20:08.026414+0000","last_active":"2024-03-07T07:20:09.575628+0000","last_peered":"2024-03-07T07:20:09.575628+0000","last_clean":"2024-03-07T07:20:09.575628+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:09.575628+0000","last_undegraded":"2024-03-07T07:20:09.575628+0000","last_fullsized":"2024-03-07T07:20:09.575628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:51.771656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576502+0000","last_change":"2024-03-07T07:20:07.027164+0000","last_active":"2024-03-07T07:20:09.576502+0000","last_peered":"2024-03-07T07:20:09.576502+0000","last_clean":"2024-03-07T07:20:09.576502+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:09.576502+0000","last_undegraded":"2024-03-07T07:20:09.576502+0000","last_fullsized":"2024-03-07T07:20:09.576502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576522+0000","last_change":"2024-03-07T07:20:08.035488+0000","last_active":"2024-03-07T07:20:09.576522+0000","last_peered":"2024-03-07T07:20:09.576522+0000","last_clean":"2024-03-07T07:20:09.576522+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:09.576522+0000","last_undegraded":"2024-03-07T07:20:09.576522+0000","last_fullsized":"2024-03-07T07:20:09.576522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.198110+0000","last_change":"2024-03-07T07:20:07.027534+0000","last_active":"2024-03-07T07:20:09.198110+0000","last_peered":"2024-03-07T07:20:09.198110+0000","last_clean":"2024-03-07T07:20:09.198110+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:09.198110+0000","last_undegraded":"2024-03-07T07:20:09.198110+0000","last_fullsized":"2024-03-07T07:20:09.198110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015303+0000","last_change":"2024-03-07T07:20:08.036269+0000","last_active":"2024-03-07T07:20:09.015303+0000","last_peered":"2024-03-07T07:20:09.015303+0000","last_clean":"2024-03-07T07:20:09.015303+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:09.015303+0000","last_undegraded":"2024-03-07T07:20:09.015303+0000","last_fullsized":"2024-03-07T07:20:09.015303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576245+0000","last_change":"2024-03-07T07:20:07.023056+0000","last_active":"2024-03-07T07:20:09.576245+0000","last_peered":"2024-03-07T07:20:09.576245+0000","last_clean":"2024-03-07T07:20:09.576245+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:09.576245+0000","last_undegraded":"2024-03-07T07:20:09.576245+0000","last_fullsized":"2024-03-07T07:20:09.576245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018197+0000","last_change":"2024-03-07T07:20:08.023363+0000","last_active":"2024-03-07T07:20:09.018197+0000","last_peered":"2024-03-07T07:20:09.018197+0000","last_clean":"2024-03-07T07:20:09.018197+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:09.018197+0000","last_undegraded":"2024-03-07T07:20:09.018197+0000","last_fullsized":"2024-03-07T07:20:09.018197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015478+0000","last_change":"2024-03-07T07:20:07.024131+0000","last_active":"2024-03-07T07:20:09.015478+0000","last_peered":"2024-03-07T07:20:09.015478+0000","last_clean":"2024-03-07T07:20:09.015478+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:09.015478+0000","last_undegraded":"2024-03-07T07:20:09.015478+0000","last_fullsized":"2024-03-07T07:20:09.015478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015390+0000","last_change":"2024-03-07T07:20:08.030007+0000","last_active":"2024-03-07T07:20:09.015390+0000","last_peered":"2024-03-07T07:20:09.015390+0000","last_clean":"2024-03-07T07:20:09.015390+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:09.015390+0000","last_undegraded":"2024-03-07T07:20:09.015390+0000","last_fullsized":"2024-03-07T07:20:09.015390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576311+0000","last_change":"2024-03-07T07:20:07.025544+0000","last_active":"2024-03-07T07:20:09.576311+0000","last_peered":"2024-03-07T07:20:09.576311+0000","last_clean":"2024-03-07T07:20:09.576311+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:09.576311+0000","last_undegraded":"2024-03-07T07:20:09.576311+0000","last_fullsized":"2024-03-07T07:20:09.576311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:18.193822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516464+0000","last_change":"2024-03-07T07:20:08.035274+0000","last_active":"2024-03-07T07:20:09.516464+0000","last_peered":"2024-03-07T07:20:09.516464+0000","last_clean":"2024-03-07T07:20:09.516464+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:09.516464+0000","last_undegraded":"2024-03-07T07:20:09.516464+0000","last_fullsized":"2024-03-07T07:20:09.516464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017946+0000","last_change":"2024-03-07T07:20:07.023385+0000","last_active":"2024-03-07T07:20:09.017946+0000","last_peered":"2024-03-07T07:20:09.017946+0000","last_clean":"2024-03-07T07:20:09.017946+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:09.017946+0000","last_undegraded":"2024-03-07T07:20:09.017946+0000","last_fullsized":"2024-03-07T07:20:09.017946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017838+0000","last_change":"2024-03-07T07:20:08.040036+0000","last_active":"2024-03-07T07:20:09.017838+0000","last_peered":"2024-03-07T07:20:09.017838+0000","last_clean":"2024-03-07T07:20:09.017838+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:09.017838+0000","last_undegraded":"2024-03-07T07:20:09.017838+0000","last_fullsized":"2024-03-07T07:20:09.017838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516475+0000","last_change":"2024-03-07T07:20:07.036824+0000","last_active":"2024-03-07T07:20:09.516475+0000","last_peered":"2024-03-07T07:20:09.516475+0000","last_clean":"2024-03-07T07:20:09.516475+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:09.516475+0000","last_undegraded":"2024-03-07T07:20:09.516475+0000","last_fullsized":"2024-03-07T07:20:09.516475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:20.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576659+0000","last_change":"2024-03-07T07:20:08.034942+0000","last_active":"2024-03-07T07:20:09.576659+0000","last_peered":"2024-03-07T07:20:09.576659+0000","last_clean":"2024-03-07T07:20:09.576659+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:09.576659+0000","last_undegraded":"2024-03-07T07:20:09.576659+0000","last_fullsized":"2024-03-07T07:20:09.576659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017654+0000","last_change":"2024-03-07T07:20:07.023606+0000","last_active":"2024-03-07T07:20:09.017654+0000","last_peered":"2024-03-07T07:20:09.017654+0000","last_clean":"2024-03-07T07:20:09.017654+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:09.017654+0000","last_undegraded":"2024-03-07T07:20:09.017654+0000","last_fullsized":"2024-03-07T07:20:09.017654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:20:19.551 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:20:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.688+0000 7f4f965d6640 1 Processor -- start 2024-03-07T07:20:19.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.688+0000 7f4f965d6640 1 -- start start 2024-03-07T07:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f965d6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4f90071c60 0x7f4f90165c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f965d6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f900b4c90 con 0x7f4f9015ad00 2024-03-07T07:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f965d6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f900b4f70 con 0x7f4f900b46b0 2024-03-07T07:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f965d6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f900b58a0 con 0x7f4f90071c60 2024-03-07T07:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f4f9015ad00 0x7f4f9006ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60214/0 (socket says 172.21.15.64:60214) 2024-03-07T07:20:19.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8ffff640 1 -- 172.21.15.64:0/1007435532 learned_addr learned my addr 172.21.15.64:0/1007435532 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f94b73640 1 --2- 172.21.15.64:0/1007435532 >> v2:172.21.15.120:3300/0 conn(0x7f4f90071c60 0x7f4f90165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046301735 0 0) 0x7f4f900b4c90 con 0x7f4f9015ad00 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c003520 con 0x7f4f9015ad00 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533433427 0 0) 0x7f4f900b4f70 con 0x7f4f900b46b0 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f900b4c90 con 0x7f4f900b46b0 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3206919259 0 0) 0x7f4f7c003520 con 0x7f4f9015ad00 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 >> v2:172.21.15.120:3300/0 conn(0x7f4f90071c60 msgr2=0x7f4f90165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 --2- 172.21.15.64:0/1007435532 >> v2:172.21.15.120:3300/0 conn(0x7f4f90071c60 0x7f4f90165c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 >> v1:172.21.15.120:6789/0 conn(0x7f4f900b46b0 legacy=0x7f4f90071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900b5250 con 0x7f4f9015ad00 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f4f80001670 con 0x7f4f9015ad00 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 >> v1:172.21.15.64:6789/0 conn(0x7f4f9015ad00 legacy=0x7f4f9006ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f7c003b40 0x7f4f7c003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f7c004800 0x7f4f7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.689+0000 7f4f8effd640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c0051c0 0x7f4f7c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900b58a0 con 0x7f4f7c003b40 2024-03-07T07:20:19.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c00e270 con 0x7f4f7c004800 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c00e470 con 0x7f4f7c0051c0 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f94b73640 1 --2- 172.21.15.64:0/1007435532 >> v2:172.21.15.120:3300/0 conn(0x7f4f90071c60 0x7f4f90165c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8ffff640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f7c003b40 0x7f4f7c003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8f7fe640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f7c004800 0x7f4f7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f94b73640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c0051c0 0x7f4f7c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8ffff640 1 -- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c0051c0 msgr2=0x7f4f7c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8ffff640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c0051c0 0x7f4f7c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8ffff640 1 -- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f7c004800 msgr2=0x7f4f7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:19.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8ffff640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f7c004800 0x7f4f7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f8ffff640 1 -- 172.21.15.64:0/1007435532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c003520 con 0x7f4f7c003b40 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.690+0000 7f4f94b73640 1 -- 172.21.15.64:0/1007435532 reap_dead start 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f8ffff640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f7c003b40 0x7f4f7c003f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007fd0 tx=0x7f4f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314c1b5031fa8afd server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80007ca0 con 0x7f4f7c003b40 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f80030070 con 0x7f4f7c003b40 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f8effd640 1 -- 172.21.15.64:0/1007435532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80005d70 con 0x7f4f7c003b40 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f965d6640 1 -- 172.21.15.64:0/1007435532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f7c003b40 msgr2=0x7f4f7c003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f965d6640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f7c003b40 0x7f4f7c003f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007fd0 tx=0x7f4f7c005830 comp rx=0 tx=0).stop 2024-03-07T07:20:19.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f965d6640 1 -- 172.21.15.64:0/1007435532 shutdown_connections 2024-03-07T07:20:19.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f965d6640 1 --2- 172.21.15.64:0/1007435532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f7c003b40 0x7f4f7c003f60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f965d6640 1 -- 172.21.15.64:0/1007435532 >> 172.21.15.64:0/1007435532 conn(0x7f4f9006b040 msgr2=0x7f4f9006b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:19.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f965d6640 1 -- 172.21.15.64:0/1007435532 shutdown_connections 2024-03-07T07:20:19.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.691+0000 7f4f965d6640 1 -- 172.21.15.64:0/1007435532 wait complete. 2024-03-07T07:20:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.692+0000 7f4f965d6640 1 Processor -- start 2024-03-07T07:20:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f965d6640 1 -- start start 2024-03-07T07:20:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f965d6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c003b40 0x7f4f90148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f965d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f901f3b40 0x7f4f90159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f965d6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f900709e0 0x7f4f90070e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f965d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9006ecf0 con 0x7f4f901f3b40 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f965d6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9015b0e0 con 0x7f4f900709e0 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f965d6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9006ee70 con 0x7f4f7c003b40 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f8f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f901f3b40 0x7f4f90159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.693+0000 7f4f8f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f901f3b40 0x7f4f90159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60532/0 (socket says 172.21.15.64:60532) 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8f7fe640 1 -- 172.21.15.64:0/363161387 learned_addr learned my addr 172.21.15.64:0/363161387 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:19.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f94b73640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f900709e0 0x7f4f90070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8ffff640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c003b40 0x7f4f90148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8f7fe640 1 -- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c003b40 msgr2=0x7f4f90148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8f7fe640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c003b40 0x7f4f90148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8f7fe640 1 -- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f900709e0 msgr2=0x7f4f90070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8f7fe640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f900709e0 0x7f4f90070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8f7fe640 1 -- 172.21.15.64:0/363161387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f901592b0 con 0x7f4f901f3b40 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8f7fe640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f901f3b40 0x7f4f90159b10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800f7a0 tx=0x7f4f7800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8cff9640 1 -- 172.21.15.64:0/363161387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7801a070 con 0x7f4f901f3b40 2024-03-07T07:20:19.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8cff9640 1 -- 172.21.15.64:0/363161387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7800fe50 con 0x7f4f901f3b40 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f8cff9640 1 -- 172.21.15.64:0/363161387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78007050 con 0x7f4f901f3b40 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f90159590 con 0x7f4f901f3b40 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.694+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f90156fa0 con 0x7f4f901f3b40 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.696+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9006f200 con 0x7f4f901f3b40 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.696+0000 7f4f8cff9640 1 -- 172.21.15.64:0/363161387 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4f780071f0 con 0x7f4f901f3b40 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.696+0000 7f4f8cff9640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7006b4d0 0x7f4f7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.696+0000 7f4f8cff9640 1 -- 172.21.15.64:0/363161387 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f78089840 con 0x7f4f901f3b40 2024-03-07T07:20:19.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.696+0000 7f4f8ffff640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7006b4d0 0x7f4f7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.697+0000 7f4f8ffff640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7006b4d0 0x7f4f7006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f80034000 tx=0x7f4f80007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:19.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.700+0000 7f4f8cff9640 1 -- 172.21.15.64:0/363161387 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4f9006f200 con 0x7f4f901f3b40 2024-03-07T07:20:19.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.896+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f901f4100 con 0x7f4f7006b4d0 2024-03-07T07:20:19.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.902+0000 7f4f8cff9640 1 -- 172.21.15.64:0/363161387 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251876 (secure 0 0 0) 0x7f4f901f4100 con 0x7f4f7006b4d0 2024-03-07T07:20:19.904 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:19.906 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:20:19.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7006b4d0 msgr2=0x7f4f7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7006b4d0 0x7f4f7006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f80034000 tx=0x7f4f80007000 comp rx=0 tx=0).stop 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f901f3b40 msgr2=0x7f4f90159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f901f3b40 0x7f4f90159b10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800f7a0 tx=0x7f4f7800fc60 comp rx=0 tx=0).stop 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 shutdown_connections 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f4f7006b4d0 0x7f4f7006d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f901f3b40 0x7f4f90159b10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f900709e0 0x7f4f90070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 --2- 172.21.15.64:0/363161387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f7c003b40 0x7f4f90148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:19.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.904+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 >> 172.21.15.64:0/363161387 conn(0x7f4f9006b040 msgr2=0x7f4f900b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.905+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 shutdown_connections 2024-03-07T07:20:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:19.905+0000 7f4f965d6640 1 -- 172.21.15.64:0/363161387 wait complete. 2024-03-07T07:20:19.915 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-03-07T07:20:18.191640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593034,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4235264,"data_stored":2358484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002119"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018002+0000","last_change":"2024-03-07T07:20:08.025425+0000","last_active":"2024-03-07T07:20:09.018002+0000","last_peered":"2024-03-07T07:20:09.018002+0000","last_clean":"2024-03-07T07:20:09.018002+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:09.018002+0000","last_undegraded":"2024-03-07T07:20:09.018002+0000","last_fullsized":"2024-03-07T07:20:09.018002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516922+0000","last_change":"2024-03-07T07:20:08.036714+0000","last_active":"2024-03-07T07:20:09.516922+0000","last_peered":"2024-03-07T07:20:09.516922+0000","last_clean":"2024-03-07T07:20:09.516922+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:09.516922+0000","last_undegraded":"2024-03-07T07:20:09.516922+0000","last_fullsized":"2024-03-07T07:20:09.516922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:43.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576955+0000","last_change":"2024-03-07T07:20:08.037973+0000","last_active":"2024-03-07T07:20:09.576955+0000","last_peered":"2024-03-07T07:20:09.576955+0000","last_clean":"2024-03-07T07:20:09.576955+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:09.576955+0000","last_undegraded":"2024-03-07T07:20:09.576955+0000","last_fullsized":"2024-03-07T07:20:09.576955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575962+0000","last_change":"2024-03-07T07:20:08.042993+0000","last_active":"2024-03-07T07:20:09.575962+0000","last_peered":"2024-03-07T07:20:09.575962+0000","last_clean":"2024-03-07T07:20:09.575962+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:09.575962+0000","last_undegraded":"2024-03-07T07:20:09.575962+0000","last_fullsized":"2024-03-07T07:20:09.575962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575521+0000","last_change":"2024-03-07T07:20:08.037872+0000","last_active":"2024-03-07T07:20:09.575521+0000","last_peered":"2024-03-07T07:20:09.575521+0000","last_clean":"2024-03-07T07:20:09.575521+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:09.575521+0000","last_undegraded":"2024-03-07T07:20:09.575521+0000","last_fullsized":"2024-03-07T07:20:09.575521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:47.883124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516617+0000","last_change":"2024-03-07T07:20:08.028212+0000","last_active":"2024-03-07T07:20:09.516617+0000","last_peered":"2024-03-07T07:20:09.516617+0000","last_clean":"2024-03-07T07:20:09.516617+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:09.516617+0000","last_undegraded":"2024-03-07T07:20:09.516617+0000","last_fullsized":"2024-03-07T07:20:09.516617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577021+0000","last_change":"2024-03-07T07:20:08.032379+0000","last_active":"2024-03-07T07:20:09.577021+0000","last_peered":"2024-03-07T07:20:09.577021+0000","last_clean":"2024-03-07T07:20:09.577021+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:09.577021+0000","last_undegraded":"2024-03-07T07:20:09.577021+0000","last_fullsized":"2024-03-07T07:20:09.577021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576904+0000","last_change":"2024-03-07T07:20:08.035495+0000","last_active":"2024-03-07T07:20:09.576904+0000","last_peered":"2024-03-07T07:20:09.576904+0000","last_clean":"2024-03-07T07:20:09.576904+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:09.576904+0000","last_undegraded":"2024-03-07T07:20:09.576904+0000","last_fullsized":"2024-03-07T07:20:09.576904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018731+0000","last_change":"2024-03-07T07:20:08.034012+0000","last_active":"2024-03-07T07:20:09.018731+0000","last_peered":"2024-03-07T07:20:09.018731+0000","last_clean":"2024-03-07T07:20:09.018731+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:09.018731+0000","last_undegraded":"2024-03-07T07:20:09.018731+0000","last_fullsized":"2024-03-07T07:20:09.018731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576070+0000","last_change":"2024-03-07T07:20:08.037279+0000","last_active":"2024-03-07T07:20:09.576070+0000","last_peered":"2024-03-07T07:20:09.576070+0000","last_clean":"2024-03-07T07:20:09.576070+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:09.576070+0000","last_undegraded":"2024-03-07T07:20:09.576070+0000","last_fullsized":"2024-03-07T07:20:09.576070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576985+0000","last_change":"2024-03-07T07:20:08.039682+0000","last_active":"2024-03-07T07:20:09.576985+0000","last_peered":"2024-03-07T07:20:09.576985+0000","last_clean":"2024-03-07T07:20:09.576985+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:09.576985+0000","last_undegraded":"2024-03-07T07:20:09.576985+0000","last_fullsized":"2024-03-07T07:20:09.576985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576401+0000","last_change":"2024-03-07T07:20:08.028305+0000","last_active":"2024-03-07T07:20:09.576401+0000","last_peered":"2024-03-07T07:20:09.576401+0000","last_clean":"2024-03-07T07:20:09.576401+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:09.576401+0000","last_undegraded":"2024-03-07T07:20:09.576401+0000","last_fullsized":"2024-03-07T07:20:09.576401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575664+0000","last_change":"2024-03-07T07:20:08.033602+0000","last_active":"2024-03-07T07:20:09.575664+0000","last_peered":"2024-03-07T07:20:09.575664+0000","last_clean":"2024-03-07T07:20:09.575664+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:09.575664+0000","last_undegraded":"2024-03-07T07:20:09.575664+0000","last_fullsized":"2024-03-07T07:20:09.575664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:55.373070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516717+0000","last_change":"2024-03-07T07:20:08.022351+0000","last_active":"2024-03-07T07:20:09.516717+0000","last_peered":"2024-03-07T07:20:09.516717+0000","last_clean":"2024-03-07T07:20:09.516717+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:09.516717+0000","last_undegraded":"2024-03-07T07:20:09.516717+0000","last_fullsized":"2024-03-07T07:20:09.516717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:20:08.021148+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577186+0000","last_change":"2024-03-07T07:20:08.035791+0000","last_active":"2024-03-07T07:20:09.577186+0000","last_peered":"2024-03-07T07:20:09.577186+0000","last_clean":"2024-03-07T07:20:09.577186+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:09.577186+0000","last_undegraded":"2024-03-07T07:20:09.577186+0000","last_fullsized":"2024-03-07T07:20:09.577186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576963+0000","last_change":"2024-03-07T07:20:08.034016+0000","last_active":"2024-03-07T07:20:09.576963+0000","last_peered":"2024-03-07T07:20:09.576963+0000","last_clean":"2024-03-07T07:20:09.576963+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:09.576963+0000","last_undegraded":"2024-03-07T07:20:09.576963+0000","last_fullsized":"2024-03-07T07:20:09.576963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517000+0000","last_change":"2024-03-07T07:20:08.035489+0000","last_active":"2024-03-07T07:20:09.517000+0000","last_peered":"2024-03-07T07:20:09.517000+0000","last_clean":"2024-03-07T07:20:09.517000+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:09.517000+0000","last_undegraded":"2024-03-07T07:20:09.517000+0000","last_fullsized":"2024-03-07T07:20:09.517000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:07.455855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516748+0000","last_change":"2024-03-07T07:20:08.037898+0000","last_active":"2024-03-07T07:20:09.516748+0000","last_peered":"2024-03-07T07:20:09.516748+0000","last_clean":"2024-03-07T07:20:09.516748+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:09.516748+0000","last_undegraded":"2024-03-07T07:20:09.516748+0000","last_fullsized":"2024-03-07T07:20:09.516748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575696+0000","last_change":"2024-03-07T07:20:08.025458+0000","last_active":"2024-03-07T07:20:09.575696+0000","last_peered":"2024-03-07T07:20:09.575696+0000","last_clean":"2024-03-07T07:20:09.575696+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:09.575696+0000","last_undegraded":"2024-03-07T07:20:09.575696+0000","last_fullsized":"2024-03-07T07:20:09.575696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:24.155245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018155+0000","last_change":"2024-03-07T07:20:08.040577+0000","last_active":"2024-03-07T07:20:09.018155+0000","last_peered":"2024-03-07T07:20:09.018155+0000","last_clean":"2024-03-07T07:20:09.018155+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:09.018155+0000","last_undegraded":"2024-03-07T07:20:09.018155+0000","last_fullsized":"2024-03-07T07:20:09.018155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018038+0000","last_change":"2024-03-07T07:20:08.039803+0000","last_active":"2024-03-07T07:20:09.018038+0000","last_peered":"2024-03-07T07:20:09.018038+0000","last_clean":"2024-03-07T07:20:09.018038+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:09.018038+0000","last_undegraded":"2024-03-07T07:20:09.018038+0000","last_fullsized":"2024-03-07T07:20:09.018038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015233+0000","last_change":"2024-03-07T07:20:08.031321+0000","last_active":"2024-03-07T07:20:09.015233+0000","last_peered":"2024-03-07T07:20:09.015233+0000","last_clean":"2024-03-07T07:20:09.015233+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:09.015233+0000","last_undegraded":"2024-03-07T07:20:09.015233+0000","last_fullsized":"2024-03-07T07:20:09.015233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576999+0000","last_change":"2024-03-07T07:20:08.024172+0000","last_active":"2024-03-07T07:20:09.576999+0000","last_peered":"2024-03-07T07:20:09.576999+0000","last_clean":"2024-03-07T07:20:09.576999+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:09.576999+0000","last_undegraded":"2024-03-07T07:20:09.576999+0000","last_fullsized":"2024-03-07T07:20:09.576999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576283+0000","last_change":"2024-03-07T07:20:08.033067+0000","last_active":"2024-03-07T07:20:09.576283+0000","last_peered":"2024-03-07T07:20:09.576283+0000","last_clean":"2024-03-07T07:20:09.576283+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:09.576283+0000","last_undegraded":"2024-03-07T07:20:09.576283+0000","last_fullsized":"2024-03-07T07:20:09.576283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517079+0000","last_change":"2024-03-07T07:20:08.031519+0000","last_active":"2024-03-07T07:20:09.517079+0000","last_peered":"2024-03-07T07:20:09.517079+0000","last_clean":"2024-03-07T07:20:09.517079+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:09.517079+0000","last_undegraded":"2024-03-07T07:20:09.517079+0000","last_fullsized":"2024-03-07T07:20:09.517079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:33.954019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576399+0000","last_change":"2024-03-07T07:20:08.043585+0000","last_active":"2024-03-07T07:20:09.576399+0000","last_peered":"2024-03-07T07:20:09.576399+0000","last_clean":"2024-03-07T07:20:09.576399+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:09.576399+0000","last_undegraded":"2024-03-07T07:20:09.576399+0000","last_fullsized":"2024-03-07T07:20:09.576399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576561+0000","last_change":"2024-03-07T07:20:08.021978+0000","last_active":"2024-03-07T07:20:09.576561+0000","last_peered":"2024-03-07T07:20:09.576561+0000","last_clean":"2024-03-07T07:20:09.576561+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:09.576561+0000","last_undegraded":"2024-03-07T07:20:09.576561+0000","last_fullsized":"2024-03-07T07:20:09.576561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018178+0000","last_change":"2024-03-07T07:20:08.040950+0000","last_active":"2024-03-07T07:20:09.018178+0000","last_peered":"2024-03-07T07:20:09.018178+0000","last_clean":"2024-03-07T07:20:09.018178+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:09.018178+0000","last_undegraded":"2024-03-07T07:20:09.018178+0000","last_fullsized":"2024-03-07T07:20:09.018178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576528+0000","last_change":"2024-03-07T07:20:08.031636+0000","last_active":"2024-03-07T07:20:09.576528+0000","last_peered":"2024-03-07T07:20:09.576528+0000","last_clean":"2024-03-07T07:20:09.576528+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:09.576528+0000","last_undegraded":"2024-03-07T07:20:09.576528+0000","last_fullsized":"2024-03-07T07:20:09.576528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017643+0000","last_change":"2024-03-07T07:20:08.023430+0000","last_active":"2024-03-07T07:20:09.017643+0000","last_peered":"2024-03-07T07:20:09.017643+0000","last_clean":"2024-03-07T07:20:09.017643+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:09.017643+0000","last_undegraded":"2024-03-07T07:20:09.017643+0000","last_fullsized":"2024-03-07T07:20:09.017643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:29.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576960+0000","last_change":"2024-03-07T07:20:08.033808+0000","last_active":"2024-03-07T07:20:09.576960+0000","last_peered":"2024-03-07T07:20:09.576960+0000","last_clean":"2024-03-07T07:20:09.576960+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:09.576960+0000","last_undegraded":"2024-03-07T07:20:09.576960+0000","last_fullsized":"2024-03-07T07:20:09.576960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015411+0000","last_change":"2024-03-07T07:20:08.030548+0000","last_active":"2024-03-07T07:20:09.015411+0000","last_peered":"2024-03-07T07:20:09.015411+0000","last_clean":"2024-03-07T07:20:09.015411+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:09.015411+0000","last_undegraded":"2024-03-07T07:20:09.015411+0000","last_fullsized":"2024-03-07T07:20:09.015411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018115+0000","last_change":"2024-03-07T07:20:07.025352+0000","last_active":"2024-03-07T07:20:09.018115+0000","last_peered":"2024-03-07T07:20:09.018115+0000","last_clean":"2024-03-07T07:20:09.018115+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:09.018115+0000","last_undegraded":"2024-03-07T07:20:09.018115+0000","last_fullsized":"2024-03-07T07:20:09.018115+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018004+0000","last_change":"2024-03-07T07:20:08.033169+0000","last_active":"2024-03-07T07:20:09.018004+0000","last_peered":"2024-03-07T07:20:09.018004+0000","last_clean":"2024-03-07T07:20:09.018004+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:09.018004+0000","last_undegraded":"2024-03-07T07:20:09.018004+0000","last_fullsized":"2024-03-07T07:20:09.018004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015349+0000","last_change":"2024-03-07T07:20:07.025175+0000","last_active":"2024-03-07T07:20:09.015349+0000","last_peered":"2024-03-07T07:20:09.015349+0000","last_clean":"2024-03-07T07:20:09.015349+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:09.015349+0000","last_undegraded":"2024-03-07T07:20:09.015349+0000","last_fullsized":"2024-03-07T07:20:09.015349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:23.151665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018559+0000","last_change":"2024-03-07T07:20:08.040222+0000","last_active":"2024-03-07T07:20:09.018559+0000","last_peered":"2024-03-07T07:20:09.018559+0000","last_clean":"2024-03-07T07:20:09.018559+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:09.018559+0000","last_undegraded":"2024-03-07T07:20:09.018559+0000","last_fullsized":"2024-03-07T07:20:09.018559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516719+0000","last_change":"2024-03-07T07:20:07.033158+0000","last_active":"2024-03-07T07:20:09.516719+0000","last_peered":"2024-03-07T07:20:09.516719+0000","last_clean":"2024-03-07T07:20:09.516719+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:09.516719+0000","last_undegraded":"2024-03-07T07:20:09.516719+0000","last_fullsized":"2024-03-07T07:20:09.516719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:30.672111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576436+0000","last_change":"2024-03-07T07:20:08.043374+0000","last_active":"2024-03-07T07:20:09.576436+0000","last_peered":"2024-03-07T07:20:09.576436+0000","last_clean":"2024-03-07T07:20:09.576436+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:09.576436+0000","last_undegraded":"2024-03-07T07:20:09.576436+0000","last_fullsized":"2024-03-07T07:20:09.576436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575854+0000","last_change":"2024-03-07T07:20:07.028373+0000","last_active":"2024-03-07T07:20:09.575854+0000","last_peered":"2024-03-07T07:20:09.575854+0000","last_clean":"2024-03-07T07:20:09.575854+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:09.575854+0000","last_undegraded":"2024-03-07T07:20:09.575854+0000","last_fullsized":"2024-03-07T07:20:09.575854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:51.229977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576660+0000","last_change":"2024-03-07T07:20:08.033882+0000","last_active":"2024-03-07T07:20:09.576660+0000","last_peered":"2024-03-07T07:20:09.576660+0000","last_clean":"2024-03-07T07:20:09.576660+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:09.576660+0000","last_undegraded":"2024-03-07T07:20:09.576660+0000","last_fullsized":"2024-03-07T07:20:09.576660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018255+0000","last_change":"2024-03-07T07:20:07.028711+0000","last_active":"2024-03-07T07:20:09.018255+0000","last_peered":"2024-03-07T07:20:09.018255+0000","last_clean":"2024-03-07T07:20:09.018255+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:09.018255+0000","last_undegraded":"2024-03-07T07:20:09.018255+0000","last_fullsized":"2024-03-07T07:20:09.018255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:46.879544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576378+0000","last_change":"2024-03-07T07:20:08.030098+0000","last_active":"2024-03-07T07:20:09.576378+0000","last_peered":"2024-03-07T07:20:09.576378+0000","last_clean":"2024-03-07T07:20:09.576378+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:09.576378+0000","last_undegraded":"2024-03-07T07:20:09.576378+0000","last_fullsized":"2024-03-07T07:20:09.576378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018003+0000","last_change":"2024-03-07T07:20:07.029834+0000","last_active":"2024-03-07T07:20:09.018003+0000","last_peered":"2024-03-07T07:20:09.018003+0000","last_clean":"2024-03-07T07:20:09.018003+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:09.018003+0000","last_undegraded":"2024-03-07T07:20:09.018003+0000","last_fullsized":"2024-03-07T07:20:09.018003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:55:49.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018248+0000","last_change":"2024-03-07T07:20:08.028053+0000","last_active":"2024-03-07T07:20:09.018248+0000","last_peered":"2024-03-07T07:20:09.018248+0000","last_clean":"2024-03-07T07:20:09.018248+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:09.018248+0000","last_undegraded":"2024-03-07T07:20:09.018248+0000","last_fullsized":"2024-03-07T07:20:09.018248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516477+0000","last_change":"2024-03-07T07:20:07.033411+0000","last_active":"2024-03-07T07:20:09.516477+0000","last_peered":"2024-03-07T07:20:09.516477+0000","last_clean":"2024-03-07T07:20:09.516477+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:09.516477+0000","last_undegraded":"2024-03-07T07:20:09.516477+0000","last_fullsized":"2024-03-07T07:20:09.516477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:15.200144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576038+0000","last_change":"2024-03-07T07:20:08.032643+0000","last_active":"2024-03-07T07:20:09.576038+0000","last_peered":"2024-03-07T07:20:09.576038+0000","last_clean":"2024-03-07T07:20:09.576038+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:09.576038+0000","last_undegraded":"2024-03-07T07:20:09.576038+0000","last_fullsized":"2024-03-07T07:20:09.576038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018364+0000","last_change":"2024-03-07T07:20:07.027022+0000","last_active":"2024-03-07T07:20:09.018364+0000","last_peered":"2024-03-07T07:20:09.018364+0000","last_clean":"2024-03-07T07:20:09.018364+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:09.018364+0000","last_undegraded":"2024-03-07T07:20:09.018364+0000","last_fullsized":"2024-03-07T07:20:09.018364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:35.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.028592+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516961+0000","last_change":"2024-03-07T07:20:07.030367+0000","last_active":"2024-03-07T07:20:09.516961+0000","last_peered":"2024-03-07T07:20:09.516961+0000","last_clean":"2024-03-07T07:20:09.516961+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:09.516961+0000","last_undegraded":"2024-03-07T07:20:09.516961+0000","last_fullsized":"2024-03-07T07:20:09.516961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:12.128114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516860+0000","last_change":"2024-03-07T07:20:08.034949+0000","last_active":"2024-03-07T07:20:09.516860+0000","last_peered":"2024-03-07T07:20:09.516860+0000","last_clean":"2024-03-07T07:20:09.516860+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:09.516860+0000","last_undegraded":"2024-03-07T07:20:09.516860+0000","last_fullsized":"2024-03-07T07:20:09.516860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015409+0000","last_change":"2024-03-07T07:20:07.030049+0000","last_active":"2024-03-07T07:20:09.015409+0000","last_peered":"2024-03-07T07:20:09.015409+0000","last_clean":"2024-03-07T07:20:09.015409+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:09.015409+0000","last_undegraded":"2024-03-07T07:20:09.015409+0000","last_fullsized":"2024-03-07T07:20:09.015409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516872+0000","last_change":"2024-03-07T07:20:08.030884+0000","last_active":"2024-03-07T07:20:09.516872+0000","last_peered":"2024-03-07T07:20:09.516872+0000","last_clean":"2024-03-07T07:20:09.516872+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:09.516872+0000","last_undegraded":"2024-03-07T07:20:09.516872+0000","last_fullsized":"2024-03-07T07:20:09.516872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:22.617306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577049+0000","last_change":"2024-03-07T07:20:07.028526+0000","last_active":"2024-03-07T07:20:09.577049+0000","last_peered":"2024-03-07T07:20:09.577049+0000","last_clean":"2024-03-07T07:20:09.577049+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:09.577049+0000","last_undegraded":"2024-03-07T07:20:09.577049+0000","last_fullsized":"2024-03-07T07:20:09.577049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576521+0000","last_change":"2024-03-07T07:20:08.038594+0000","last_active":"2024-03-07T07:20:09.576521+0000","last_peered":"2024-03-07T07:20:09.576521+0000","last_clean":"2024-03-07T07:20:09.576521+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:09.576521+0000","last_undegraded":"2024-03-07T07:20:09.576521+0000","last_fullsized":"2024-03-07T07:20:09.576521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'7","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516823+0000","last_change":"2024-03-07T07:20:07.035824+0000","last_active":"2024-03-07T07:20:09.516823+0000","last_peered":"2024-03-07T07:20:09.516823+0000","last_clean":"2024-03-07T07:20:09.516823+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:09.516823+0000","last_undegraded":"2024-03-07T07:20:09.516823+0000","last_fullsized":"2024-03-07T07:20:09.516823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576168+0000","last_change":"2024-03-07T07:20:08.034611+0000","last_active":"2024-03-07T07:20:09.576168+0000","last_peered":"2024-03-07T07:20:09.576168+0000","last_clean":"2024-03-07T07:20:09.576168+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:09.576168+0000","last_undegraded":"2024-03-07T07:20:09.576168+0000","last_fullsized":"2024-03-07T07:20:09.576168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018274+0000","last_change":"2024-03-07T07:20:07.025705+0000","last_active":"2024-03-07T07:20:09.018274+0000","last_peered":"2024-03-07T07:20:09.018274+0000","last_clean":"2024-03-07T07:20:09.018274+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:09.018274+0000","last_undegraded":"2024-03-07T07:20:09.018274+0000","last_fullsized":"2024-03-07T07:20:09.018274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516759+0000","last_change":"2024-03-07T07:20:08.037145+0000","last_active":"2024-03-07T07:20:09.516759+0000","last_peered":"2024-03-07T07:20:09.516759+0000","last_clean":"2024-03-07T07:20:09.516759+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:09.516759+0000","last_undegraded":"2024-03-07T07:20:09.516759+0000","last_fullsized":"2024-03-07T07:20:09.516759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576262+0000","last_change":"2024-03-07T07:20:07.026780+0000","last_active":"2024-03-07T07:20:09.576262+0000","last_peered":"2024-03-07T07:20:09.576262+0000","last_clean":"2024-03-07T07:20:09.576262+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:09.576262+0000","last_undegraded":"2024-03-07T07:20:09.576262+0000","last_fullsized":"2024-03-07T07:20:09.576262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577126+0000","last_change":"2024-03-07T07:20:08.033869+0000","last_active":"2024-03-07T07:20:09.577126+0000","last_peered":"2024-03-07T07:20:09.577126+0000","last_clean":"2024-03-07T07:20:09.577126+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:09.577126+0000","last_undegraded":"2024-03-07T07:20:09.577126+0000","last_fullsized":"2024-03-07T07:20:09.577126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516453+0000","last_change":"2024-03-07T07:20:07.032636+0000","last_active":"2024-03-07T07:20:09.516453+0000","last_peered":"2024-03-07T07:20:09.516453+0000","last_clean":"2024-03-07T07:20:09.516453+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:09.516453+0000","last_undegraded":"2024-03-07T07:20:09.516453+0000","last_fullsized":"2024-03-07T07:20:09.516453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017781+0000","last_change":"2024-03-07T07:20:08.036782+0000","last_active":"2024-03-07T07:20:09.017781+0000","last_peered":"2024-03-07T07:20:09.017781+0000","last_clean":"2024-03-07T07:20:09.017781+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:09.017781+0000","last_undegraded":"2024-03-07T07:20:09.017781+0000","last_fullsized":"2024-03-07T07:20:09.017781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:19.197402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516567+0000","last_change":"2024-03-07T07:20:07.033230+0000","last_active":"2024-03-07T07:20:09.516567+0000","last_peered":"2024-03-07T07:20:09.516567+0000","last_clean":"2024-03-07T07:20:09.516567+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:09.516567+0000","last_undegraded":"2024-03-07T07:20:09.516567+0000","last_fullsized":"2024-03-07T07:20:09.516567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:42.777795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018625+0000","last_change":"2024-03-07T07:20:08.036934+0000","last_active":"2024-03-07T07:20:09.018625+0000","last_peered":"2024-03-07T07:20:09.018625+0000","last_clean":"2024-03-07T07:20:09.018625+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:09.018625+0000","last_undegraded":"2024-03-07T07:20:09.018625+0000","last_fullsized":"2024-03-07T07:20:09.018625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576460+0000","last_change":"2024-03-07T07:20:07.033018+0000","last_active":"2024-03-07T07:20:09.576460+0000","last_peered":"2024-03-07T07:20:09.576460+0000","last_clean":"2024-03-07T07:20:09.576460+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:09.576460+0000","last_undegraded":"2024-03-07T07:20:09.576460+0000","last_fullsized":"2024-03-07T07:20:09.576460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577155+0000","last_change":"2024-03-07T07:20:08.035720+0000","last_active":"2024-03-07T07:20:09.577155+0000","last_peered":"2024-03-07T07:20:09.577155+0000","last_clean":"2024-03-07T07:20:09.577155+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:09.577155+0000","last_undegraded":"2024-03-07T07:20:09.577155+0000","last_fullsized":"2024-03-07T07:20:09.577155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018650+0000","last_change":"2024-03-07T07:20:07.026560+0000","last_active":"2024-03-07T07:20:09.018650+0000","last_peered":"2024-03-07T07:20:09.018650+0000","last_clean":"2024-03-07T07:20:09.018650+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:09.018650+0000","last_undegraded":"2024-03-07T07:20:09.018650+0000","last_fullsized":"2024-03-07T07:20:09.018650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:52.614111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576186+0000","last_change":"2024-03-07T07:20:08.033852+0000","last_active":"2024-03-07T07:20:09.576186+0000","last_peered":"2024-03-07T07:20:09.576186+0000","last_clean":"2024-03-07T07:20:09.576186+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:09.576186+0000","last_undegraded":"2024-03-07T07:20:09.576186+0000","last_fullsized":"2024-03-07T07:20:09.576186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018510+0000","last_change":"2024-03-07T07:20:07.023668+0000","last_active":"2024-03-07T07:20:09.018510+0000","last_peered":"2024-03-07T07:20:09.018510+0000","last_clean":"2024-03-07T07:20:09.018510+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:09.018510+0000","last_undegraded":"2024-03-07T07:20:09.018510+0000","last_fullsized":"2024-03-07T07:20:09.018510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:07.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575815+0000","last_change":"2024-03-07T07:20:08.030524+0000","last_active":"2024-03-07T07:20:09.575815+0000","last_peered":"2024-03-07T07:20:09.575815+0000","last_clean":"2024-03-07T07:20:09.575815+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:09.575815+0000","last_undegraded":"2024-03-07T07:20:09.575815+0000","last_fullsized":"2024-03-07T07:20:09.575815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:18.342434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018550+0000","last_change":"2024-03-07T07:20:07.029892+0000","last_active":"2024-03-07T07:20:09.018550+0000","last_peered":"2024-03-07T07:20:09.018550+0000","last_clean":"2024-03-07T07:20:09.018550+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:09.018550+0000","last_undegraded":"2024-03-07T07:20:09.018550+0000","last_fullsized":"2024-03-07T07:20:09.018550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:24.735113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575612+0000","last_change":"2024-03-07T07:20:08.032134+0000","last_active":"2024-03-07T07:20:09.575612+0000","last_peered":"2024-03-07T07:20:09.575612+0000","last_clean":"2024-03-07T07:20:09.575612+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:09.575612+0000","last_undegraded":"2024-03-07T07:20:09.575612+0000","last_fullsized":"2024-03-07T07:20:09.575612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:48.726782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015375+0000","last_change":"2024-03-07T07:20:07.028153+0000","last_active":"2024-03-07T07:20:09.015375+0000","last_peered":"2024-03-07T07:20:09.015375+0000","last_clean":"2024-03-07T07:20:09.015375+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:09.015375+0000","last_undegraded":"2024-03-07T07:20:09.015375+0000","last_fullsized":"2024-03-07T07:20:09.015375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575806+0000","last_change":"2024-03-07T07:20:08.032594+0000","last_active":"2024-03-07T07:20:09.575806+0000","last_peered":"2024-03-07T07:20:09.575806+0000","last_clean":"2024-03-07T07:20:09.575806+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:09.575806+0000","last_undegraded":"2024-03-07T07:20:09.575806+0000","last_fullsized":"2024-03-07T07:20:09.575806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:15.552375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018303+0000","last_change":"2024-03-07T07:20:07.028455+0000","last_active":"2024-03-07T07:20:09.018303+0000","last_peered":"2024-03-07T07:20:09.018303+0000","last_clean":"2024-03-07T07:20:09.018303+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:09.018303+0000","last_undegraded":"2024-03-07T07:20:09.018303+0000","last_fullsized":"2024-03-07T07:20:09.018303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575541+0000","last_change":"2024-03-07T07:20:08.031299+0000","last_active":"2024-03-07T07:20:09.575541+0000","last_peered":"2024-03-07T07:20:09.575541+0000","last_clean":"2024-03-07T07:20:09.575541+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:09.575541+0000","last_undegraded":"2024-03-07T07:20:09.575541+0000","last_fullsized":"2024-03-07T07:20:09.575541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:24:00.520723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017852+0000","last_change":"2024-03-07T07:20:07.028369+0000","last_active":"2024-03-07T07:20:09.017852+0000","last_peered":"2024-03-07T07:20:09.017852+0000","last_clean":"2024-03-07T07:20:09.017852+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:09.017852+0000","last_undegraded":"2024-03-07T07:20:09.017852+0000","last_fullsized":"2024-03-07T07:20:09.017852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.033176+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576122+0000","last_change":"2024-03-07T07:20:07.029244+0000","last_active":"2024-03-07T07:20:09.576122+0000","last_peered":"2024-03-07T07:20:09.576122+0000","last_clean":"2024-03-07T07:20:09.576122+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:09.576122+0000","last_undegraded":"2024-03-07T07:20:09.576122+0000","last_fullsized":"2024-03-07T07:20:09.576122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575628+0000","last_change":"2024-03-07T07:20:08.026414+0000","last_active":"2024-03-07T07:20:09.575628+0000","last_peered":"2024-03-07T07:20:09.575628+0000","last_clean":"2024-03-07T07:20:09.575628+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:09.575628+0000","last_undegraded":"2024-03-07T07:20:09.575628+0000","last_fullsized":"2024-03-07T07:20:09.575628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:51.771656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576502+0000","last_change":"2024-03-07T07:20:07.027164+0000","last_active":"2024-03-07T07:20:09.576502+0000","last_peered":"2024-03-07T07:20:09.576502+0000","last_clean":"2024-03-07T07:20:09.576502+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:09.576502+0000","last_undegraded":"2024-03-07T07:20:09.576502+0000","last_fullsized":"2024-03-07T07:20:09.576502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576522+0000","last_change":"2024-03-07T07:20:08.035488+0000","last_active":"2024-03-07T07:20:09.576522+0000","last_peered":"2024-03-07T07:20:09.576522+0000","last_clean":"2024-03-07T07:20:09.576522+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:09.576522+0000","last_undegraded":"2024-03-07T07:20:09.576522+0000","last_fullsized":"2024-03-07T07:20:09.576522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.198110+0000","last_change":"2024-03-07T07:20:07.027534+0000","last_active":"2024-03-07T07:20:09.198110+0000","last_peered":"2024-03-07T07:20:09.198110+0000","last_clean":"2024-03-07T07:20:09.198110+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:09.198110+0000","last_undegraded":"2024-03-07T07:20:09.198110+0000","last_fullsized":"2024-03-07T07:20:09.198110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015303+0000","last_change":"2024-03-07T07:20:08.036269+0000","last_active":"2024-03-07T07:20:09.015303+0000","last_peered":"2024-03-07T07:20:09.015303+0000","last_clean":"2024-03-07T07:20:09.015303+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:09.015303+0000","last_undegraded":"2024-03-07T07:20:09.015303+0000","last_fullsized":"2024-03-07T07:20:09.015303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576245+0000","last_change":"2024-03-07T07:20:07.023056+0000","last_active":"2024-03-07T07:20:09.576245+0000","last_peered":"2024-03-07T07:20:09.576245+0000","last_clean":"2024-03-07T07:20:09.576245+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:09.576245+0000","last_undegraded":"2024-03-07T07:20:09.576245+0000","last_fullsized":"2024-03-07T07:20:09.576245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018197+0000","last_change":"2024-03-07T07:20:08.023363+0000","last_active":"2024-03-07T07:20:09.018197+0000","last_peered":"2024-03-07T07:20:09.018197+0000","last_clean":"2024-03-07T07:20:09.018197+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:09.018197+0000","last_undegraded":"2024-03-07T07:20:09.018197+0000","last_fullsized":"2024-03-07T07:20:09.018197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015478+0000","last_change":"2024-03-07T07:20:07.024131+0000","last_active":"2024-03-07T07:20:09.015478+0000","last_peered":"2024-03-07T07:20:09.015478+0000","last_clean":"2024-03-07T07:20:09.015478+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:09.015478+0000","last_undegraded":"2024-03-07T07:20:09.015478+0000","last_fullsized":"2024-03-07T07:20:09.015478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015390+0000","last_change":"2024-03-07T07:20:08.030007+0000","last_active":"2024-03-07T07:20:09.015390+0000","last_peered":"2024-03-07T07:20:09.015390+0000","last_clean":"2024-03-07T07:20:09.015390+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:09.015390+0000","last_undegraded":"2024-03-07T07:20:09.015390+0000","last_fullsized":"2024-03-07T07:20:09.015390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576311+0000","last_change":"2024-03-07T07:20:07.025544+0000","last_active":"2024-03-07T07:20:09.576311+0000","last_peered":"2024-03-07T07:20:09.576311+0000","last_clean":"2024-03-07T07:20:09.576311+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:09.576311+0000","last_undegraded":"2024-03-07T07:20:09.576311+0000","last_fullsized":"2024-03-07T07:20:09.576311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:18.193822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516464+0000","last_change":"2024-03-07T07:20:08.035274+0000","last_active":"2024-03-07T07:20:09.516464+0000","last_peered":"2024-03-07T07:20:09.516464+0000","last_clean":"2024-03-07T07:20:09.516464+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:09.516464+0000","last_undegraded":"2024-03-07T07:20:09.516464+0000","last_fullsized":"2024-03-07T07:20:09.516464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017946+0000","last_change":"2024-03-07T07:20:07.023385+0000","last_active":"2024-03-07T07:20:09.017946+0000","last_peered":"2024-03-07T07:20:09.017946+0000","last_clean":"2024-03-07T07:20:09.017946+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:09.017946+0000","last_undegraded":"2024-03-07T07:20:09.017946+0000","last_fullsized":"2024-03-07T07:20:09.017946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017838+0000","last_change":"2024-03-07T07:20:08.040036+0000","last_active":"2024-03-07T07:20:09.017838+0000","last_peered":"2024-03-07T07:20:09.017838+0000","last_clean":"2024-03-07T07:20:09.017838+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:09.017838+0000","last_undegraded":"2024-03-07T07:20:09.017838+0000","last_fullsized":"2024-03-07T07:20:09.017838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516475+0000","last_change":"2024-03-07T07:20:07.036824+0000","last_active":"2024-03-07T07:20:09.516475+0000","last_peered":"2024-03-07T07:20:09.516475+0000","last_clean":"2024-03-07T07:20:09.516475+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:09.516475+0000","last_undegraded":"2024-03-07T07:20:09.516475+0000","last_fullsized":"2024-03-07T07:20:09.516475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:20.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576659+0000","last_change":"2024-03-07T07:20:08.034942+0000","last_active":"2024-03-07T07:20:09.576659+0000","last_peered":"2024-03-07T07:20:09.576659+0000","last_clean":"2024-03-07T07:20:09.576659+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:09.576659+0000","last_undegraded":"2024-03-07T07:20:09.576659+0000","last_fullsized":"2024-03-07T07:20:09.576659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017654+0000","last_change":"2024-03-07T07:20:07.023606+0000","last_active":"2024-03-07T07:20:09.017654+0000","last_peered":"2024-03-07T07:20:09.017654+0000","last_clean":"2024-03-07T07:20:09.017654+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:09.017654+0000","last_undegraded":"2024-03-07T07:20:09.017654+0000","last_fullsized":"2024-03-07T07:20:09.017654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:20:19.919 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-07T07:20:19.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:20:20.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.057+0000 7fc57b51d640 1 Processor -- start 2024-03-07T07:20:20.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.057+0000 7fc57b51d640 1 -- start start 2024-03-07T07:20:20.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc57b51d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc574140ec0 0x7fc5741416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc57b51d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc574161c90 0x7fc574162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc57b51d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57415f420 con 0x7fc57415e6a0 2024-03-07T07:20:20.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc57b51d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57415f200 con 0x7fc574161c90 2024-03-07T07:20:20.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc57b51d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5740b4590 con 0x7fc574140ec0 2024-03-07T07:20:20.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc578ab8640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fc574161c90 0x7fc574162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc579aba640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fc57415e6a0 0x7fc57415eaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:60230/0 (socket says 172.21.15.64:60230) 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc579aba640 1 -- 172.21.15.64:0/1280457436 learned_addr learned my addr 172.21.15.64:0/1280457436 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc5792b9640 1 --2- 172.21.15.64:0/1280457436 >> v2:172.21.15.120:3300/0 conn(0x7fc574140ec0 0x7fc5741416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc578ab8640 1 -- 172.21.15.64:0/1280457436 >> v2:172.21.15.120:3300/0 conn(0x7fc574140ec0 msgr2=0x7fc5741416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc578ab8640 1 --2- 172.21.15.64:0/1280457436 >> v2:172.21.15.120:3300/0 conn(0x7fc574140ec0 0x7fc5741416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc578ab8640 1 -- 172.21.15.64:0/1280457436 >> v1:172.21.15.64:6789/0 conn(0x7fc57415e6a0 legacy=0x7fc57415eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc578ab8640 1 -- 172.21.15.64:0/1280457436 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5740b4870 con 0x7fc574161c90 2024-03-07T07:20:20.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.058+0000 7fc5792b9640 1 --2- 172.21.15.64:0/1280457436 >> v2:172.21.15.120:3300/0 conn(0x7fc574140ec0 0x7fc5741416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.059+0000 7fc578ab8640 1 --2- 172.21.15.64:0/1280457436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc574161c90 0x7fc574162110 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc5740d40b0 tx=0x7fc56402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf2dceec543a070 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.059+0000 7fc56bfff640 1 -- 172.21.15.64:0/1280457436 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564035020 con 0x7fc574161c90 2024-03-07T07:20:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.059+0000 7fc56bfff640 1 -- 172.21.15.64:0/1280457436 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc564031830 con 0x7fc574161c90 2024-03-07T07:20:20.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.059+0000 7fc56bfff640 1 -- 172.21.15.64:0/1280457436 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564031c00 con 0x7fc574161c90 2024-03-07T07:20:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 -- 172.21.15.64:0/1280457436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc574161c90 msgr2=0x7fc574162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 --2- 172.21.15.64:0/1280457436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc574161c90 0x7fc574162110 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc5740d40b0 tx=0x7fc56402b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 -- 172.21.15.64:0/1280457436 shutdown_connections 2024-03-07T07:20:20.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 --2- 172.21.15.64:0/1280457436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc574161c90 0x7fc574162110 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 --2- 172.21.15.64:0/1280457436 >> v2:172.21.15.120:3300/0 conn(0x7fc574140ec0 0x7fc5741416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 -- 172.21.15.64:0/1280457436 >> 172.21.15.64:0/1280457436 conn(0x7fc574071340 msgr2=0x7fc574071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 -- 172.21.15.64:0/1280457436 shutdown_connections 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.060+0000 7fc57b51d640 1 -- 172.21.15.64:0/1280457436 wait complete. 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.061+0000 7fc57b51d640 1 Processor -- start 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.061+0000 7fc57b51d640 1 -- start start 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc57b51d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc574140ec0 0x7fc574154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc57b51d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc57415e6a0 0x7fc574150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc57b51d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc574161c90 0x7fc57413dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc57b51d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc574141420 con 0x7fc57415e6a0 2024-03-07T07:20:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc57b51d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5741412a0 con 0x7fc574161c90 2024-03-07T07:20:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc57b51d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5740b5070 con 0x7fc574140ec0 2024-03-07T07:20:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc57415e6a0 0x7fc574150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc57415e6a0 0x7fc574150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60548/0 (socket says 172.21.15.64:60548) 2024-03-07T07:20:20.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 -- 172.21.15.64:0/3715577973 learned_addr learned my addr 172.21.15.64:0/3715577973 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc579aba640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc574161c90 0x7fc57413dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc5792b9640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc574140ec0 0x7fc574154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 -- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc574140ec0 msgr2=0x7fc574154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc574140ec0 0x7fc574154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 -- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc574161c90 msgr2=0x7fc57413dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc574161c90 0x7fc57413dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 -- 172.21.15.64:0/3715577973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574154da0 con 0x7fc57415e6a0 2024-03-07T07:20:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc579aba640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc574161c90 0x7fc57413dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc578ab8640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc57415e6a0 0x7fc574150f60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc564009890 tx=0x7fc5640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc569ffb640 1 -- 172.21.15.64:0/3715577973 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564040070 con 0x7fc57415e6a0 2024-03-07T07:20:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc569ffb640 1 -- 172.21.15.64:0/3715577973 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56403e070 con 0x7fc57415e6a0 2024-03-07T07:20:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc574152390 con 0x7fc57415e6a0 2024-03-07T07:20:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.062+0000 7fc569ffb640 1 -- 172.21.15.64:0/3715577973 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564035040 con 0x7fc57415e6a0 2024-03-07T07:20:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.063+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc574154060 con 0x7fc57415e6a0 2024-03-07T07:20:20.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.064+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc574062c90 con 0x7fc57415e6a0 2024-03-07T07:20:20.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.065+0000 7fc569ffb640 1 -- 172.21.15.64:0/3715577973 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc564038020 con 0x7fc57415e6a0 2024-03-07T07:20:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.065+0000 7fc569ffb640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc55806b4d0 0x7fc55806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.065+0000 7fc569ffb640 1 -- 172.21.15.64:0/3715577973 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5640abc40 con 0x7fc57415e6a0 2024-03-07T07:20:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.065+0000 7fc5792b9640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc55806b4d0 0x7fc55806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.066+0000 7fc5792b9640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc55806b4d0 0x7fc55806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc570006fd0 tx=0x7fc570008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:20.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.069+0000 7fc569ffb640 1 -- 172.21.15.64:0/3715577973 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc56407aac0 con 0x7fc57415e6a0 2024-03-07T07:20:20.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.247+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc574144280 con 0x7fc55806b4d0 2024-03-07T07:20:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.255+0000 7fc569ffb640 1 -- 172.21.15.64:0/3715577973 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251846 (secure 0 0 0) 0x7fc574144280 con 0x7fc55806b4d0 2024-03-07T07:20:20.258 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:20.259 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:20:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc55806b4d0 msgr2=0x7fc55806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc55806b4d0 0x7fc55806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc570006fd0 tx=0x7fc570008040 comp rx=0 tx=0).stop 2024-03-07T07:20:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc57415e6a0 msgr2=0x7fc574150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc57415e6a0 0x7fc574150f60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc564009890 tx=0x7fc5640046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 shutdown_connections 2024-03-07T07:20:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fc55806b4d0 0x7fc55806d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc57415e6a0 0x7fc574150f60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc574161c90 0x7fc57413dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 --2- 172.21.15.64:0/3715577973 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc574140ec0 0x7fc574154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 >> 172.21.15.64:0/3715577973 conn(0x7fc574071340 msgr2=0x7fc574155390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 shutdown_connections 2024-03-07T07:20:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.258+0000 7fc57b51d640 1 -- 172.21.15.64:0/3715577973 wait complete. 2024-03-07T07:20:20.269 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-07T07:20:20.191962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598311,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220308,"kb_used_data":4144,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754412,"statfs":{"total":773094113280,"available":772868517888,"internally_reserved":0,"allocated":4243456,"data_stored":2369550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7943,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.176752"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018002+0000","last_change":"2024-03-07T07:20:08.025425+0000","last_active":"2024-03-07T07:20:09.018002+0000","last_peered":"2024-03-07T07:20:09.018002+0000","last_clean":"2024-03-07T07:20:09.018002+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:09.018002+0000","last_undegraded":"2024-03-07T07:20:09.018002+0000","last_fullsized":"2024-03-07T07:20:09.018002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516922+0000","last_change":"2024-03-07T07:20:08.036714+0000","last_active":"2024-03-07T07:20:09.516922+0000","last_peered":"2024-03-07T07:20:09.516922+0000","last_clean":"2024-03-07T07:20:09.516922+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:09.516922+0000","last_undegraded":"2024-03-07T07:20:09.516922+0000","last_fullsized":"2024-03-07T07:20:09.516922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:43.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576955+0000","last_change":"2024-03-07T07:20:08.037973+0000","last_active":"2024-03-07T07:20:09.576955+0000","last_peered":"2024-03-07T07:20:09.576955+0000","last_clean":"2024-03-07T07:20:09.576955+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:09.576955+0000","last_undegraded":"2024-03-07T07:20:09.576955+0000","last_fullsized":"2024-03-07T07:20:09.576955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575962+0000","last_change":"2024-03-07T07:20:08.042993+0000","last_active":"2024-03-07T07:20:09.575962+0000","last_peered":"2024-03-07T07:20:09.575962+0000","last_clean":"2024-03-07T07:20:09.575962+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:09.575962+0000","last_undegraded":"2024-03-07T07:20:09.575962+0000","last_fullsized":"2024-03-07T07:20:09.575962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575521+0000","last_change":"2024-03-07T07:20:08.037872+0000","last_active":"2024-03-07T07:20:09.575521+0000","last_peered":"2024-03-07T07:20:09.575521+0000","last_clean":"2024-03-07T07:20:09.575521+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:09.575521+0000","last_undegraded":"2024-03-07T07:20:09.575521+0000","last_fullsized":"2024-03-07T07:20:09.575521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:47.883124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516617+0000","last_change":"2024-03-07T07:20:08.028212+0000","last_active":"2024-03-07T07:20:09.516617+0000","last_peered":"2024-03-07T07:20:09.516617+0000","last_clean":"2024-03-07T07:20:09.516617+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:09.516617+0000","last_undegraded":"2024-03-07T07:20:09.516617+0000","last_fullsized":"2024-03-07T07:20:09.516617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577021+0000","last_change":"2024-03-07T07:20:08.032379+0000","last_active":"2024-03-07T07:20:09.577021+0000","last_peered":"2024-03-07T07:20:09.577021+0000","last_clean":"2024-03-07T07:20:09.577021+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:09.577021+0000","last_undegraded":"2024-03-07T07:20:09.577021+0000","last_fullsized":"2024-03-07T07:20:09.577021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576904+0000","last_change":"2024-03-07T07:20:08.035495+0000","last_active":"2024-03-07T07:20:09.576904+0000","last_peered":"2024-03-07T07:20:09.576904+0000","last_clean":"2024-03-07T07:20:09.576904+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:09.576904+0000","last_undegraded":"2024-03-07T07:20:09.576904+0000","last_fullsized":"2024-03-07T07:20:09.576904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018731+0000","last_change":"2024-03-07T07:20:08.034012+0000","last_active":"2024-03-07T07:20:09.018731+0000","last_peered":"2024-03-07T07:20:09.018731+0000","last_clean":"2024-03-07T07:20:09.018731+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:09.018731+0000","last_undegraded":"2024-03-07T07:20:09.018731+0000","last_fullsized":"2024-03-07T07:20:09.018731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576070+0000","last_change":"2024-03-07T07:20:08.037279+0000","last_active":"2024-03-07T07:20:09.576070+0000","last_peered":"2024-03-07T07:20:09.576070+0000","last_clean":"2024-03-07T07:20:09.576070+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:09.576070+0000","last_undegraded":"2024-03-07T07:20:09.576070+0000","last_fullsized":"2024-03-07T07:20:09.576070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576985+0000","last_change":"2024-03-07T07:20:08.039682+0000","last_active":"2024-03-07T07:20:09.576985+0000","last_peered":"2024-03-07T07:20:09.576985+0000","last_clean":"2024-03-07T07:20:09.576985+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:09.576985+0000","last_undegraded":"2024-03-07T07:20:09.576985+0000","last_fullsized":"2024-03-07T07:20:09.576985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576401+0000","last_change":"2024-03-07T07:20:08.028305+0000","last_active":"2024-03-07T07:20:09.576401+0000","last_peered":"2024-03-07T07:20:09.576401+0000","last_clean":"2024-03-07T07:20:09.576401+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:09.576401+0000","last_undegraded":"2024-03-07T07:20:09.576401+0000","last_fullsized":"2024-03-07T07:20:09.576401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575664+0000","last_change":"2024-03-07T07:20:08.033602+0000","last_active":"2024-03-07T07:20:09.575664+0000","last_peered":"2024-03-07T07:20:09.575664+0000","last_clean":"2024-03-07T07:20:09.575664+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:09.575664+0000","last_undegraded":"2024-03-07T07:20:09.575664+0000","last_fullsized":"2024-03-07T07:20:09.575664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:55.373070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516717+0000","last_change":"2024-03-07T07:20:08.022351+0000","last_active":"2024-03-07T07:20:09.516717+0000","last_peered":"2024-03-07T07:20:09.516717+0000","last_clean":"2024-03-07T07:20:09.516717+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:09.516717+0000","last_undegraded":"2024-03-07T07:20:09.516717+0000","last_fullsized":"2024-03-07T07:20:09.516717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:20:08.021148+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577186+0000","last_change":"2024-03-07T07:20:08.035791+0000","last_active":"2024-03-07T07:20:09.577186+0000","last_peered":"2024-03-07T07:20:09.577186+0000","last_clean":"2024-03-07T07:20:09.577186+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:09.577186+0000","last_undegraded":"2024-03-07T07:20:09.577186+0000","last_fullsized":"2024-03-07T07:20:09.577186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576963+0000","last_change":"2024-03-07T07:20:08.034016+0000","last_active":"2024-03-07T07:20:09.576963+0000","last_peered":"2024-03-07T07:20:09.576963+0000","last_clean":"2024-03-07T07:20:09.576963+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:09.576963+0000","last_undegraded":"2024-03-07T07:20:09.576963+0000","last_fullsized":"2024-03-07T07:20:09.576963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517000+0000","last_change":"2024-03-07T07:20:08.035489+0000","last_active":"2024-03-07T07:20:09.517000+0000","last_peered":"2024-03-07T07:20:09.517000+0000","last_clean":"2024-03-07T07:20:09.517000+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:09.517000+0000","last_undegraded":"2024-03-07T07:20:09.517000+0000","last_fullsized":"2024-03-07T07:20:09.517000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:07.455855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516748+0000","last_change":"2024-03-07T07:20:08.037898+0000","last_active":"2024-03-07T07:20:09.516748+0000","last_peered":"2024-03-07T07:20:09.516748+0000","last_clean":"2024-03-07T07:20:09.516748+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:09.516748+0000","last_undegraded":"2024-03-07T07:20:09.516748+0000","last_fullsized":"2024-03-07T07:20:09.516748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575696+0000","last_change":"2024-03-07T07:20:08.025458+0000","last_active":"2024-03-07T07:20:09.575696+0000","last_peered":"2024-03-07T07:20:09.575696+0000","last_clean":"2024-03-07T07:20:09.575696+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:09.575696+0000","last_undegraded":"2024-03-07T07:20:09.575696+0000","last_fullsized":"2024-03-07T07:20:09.575696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:24.155245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018155+0000","last_change":"2024-03-07T07:20:08.040577+0000","last_active":"2024-03-07T07:20:09.018155+0000","last_peered":"2024-03-07T07:20:09.018155+0000","last_clean":"2024-03-07T07:20:09.018155+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:09.018155+0000","last_undegraded":"2024-03-07T07:20:09.018155+0000","last_fullsized":"2024-03-07T07:20:09.018155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018038+0000","last_change":"2024-03-07T07:20:08.039803+0000","last_active":"2024-03-07T07:20:09.018038+0000","last_peered":"2024-03-07T07:20:09.018038+0000","last_clean":"2024-03-07T07:20:09.018038+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:09.018038+0000","last_undegraded":"2024-03-07T07:20:09.018038+0000","last_fullsized":"2024-03-07T07:20:09.018038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015233+0000","last_change":"2024-03-07T07:20:08.031321+0000","last_active":"2024-03-07T07:20:09.015233+0000","last_peered":"2024-03-07T07:20:09.015233+0000","last_clean":"2024-03-07T07:20:09.015233+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:09.015233+0000","last_undegraded":"2024-03-07T07:20:09.015233+0000","last_fullsized":"2024-03-07T07:20:09.015233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576999+0000","last_change":"2024-03-07T07:20:08.024172+0000","last_active":"2024-03-07T07:20:09.576999+0000","last_peered":"2024-03-07T07:20:09.576999+0000","last_clean":"2024-03-07T07:20:09.576999+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:09.576999+0000","last_undegraded":"2024-03-07T07:20:09.576999+0000","last_fullsized":"2024-03-07T07:20:09.576999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576283+0000","last_change":"2024-03-07T07:20:08.033067+0000","last_active":"2024-03-07T07:20:09.576283+0000","last_peered":"2024-03-07T07:20:09.576283+0000","last_clean":"2024-03-07T07:20:09.576283+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:09.576283+0000","last_undegraded":"2024-03-07T07:20:09.576283+0000","last_fullsized":"2024-03-07T07:20:09.576283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517079+0000","last_change":"2024-03-07T07:20:08.031519+0000","last_active":"2024-03-07T07:20:09.517079+0000","last_peered":"2024-03-07T07:20:09.517079+0000","last_clean":"2024-03-07T07:20:09.517079+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:09.517079+0000","last_undegraded":"2024-03-07T07:20:09.517079+0000","last_fullsized":"2024-03-07T07:20:09.517079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:33.954019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576399+0000","last_change":"2024-03-07T07:20:08.043585+0000","last_active":"2024-03-07T07:20:09.576399+0000","last_peered":"2024-03-07T07:20:09.576399+0000","last_clean":"2024-03-07T07:20:09.576399+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:09.576399+0000","last_undegraded":"2024-03-07T07:20:09.576399+0000","last_fullsized":"2024-03-07T07:20:09.576399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576561+0000","last_change":"2024-03-07T07:20:08.021978+0000","last_active":"2024-03-07T07:20:09.576561+0000","last_peered":"2024-03-07T07:20:09.576561+0000","last_clean":"2024-03-07T07:20:09.576561+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:09.576561+0000","last_undegraded":"2024-03-07T07:20:09.576561+0000","last_fullsized":"2024-03-07T07:20:09.576561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018178+0000","last_change":"2024-03-07T07:20:08.040950+0000","last_active":"2024-03-07T07:20:09.018178+0000","last_peered":"2024-03-07T07:20:09.018178+0000","last_clean":"2024-03-07T07:20:09.018178+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:09.018178+0000","last_undegraded":"2024-03-07T07:20:09.018178+0000","last_fullsized":"2024-03-07T07:20:09.018178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576528+0000","last_change":"2024-03-07T07:20:08.031636+0000","last_active":"2024-03-07T07:20:09.576528+0000","last_peered":"2024-03-07T07:20:09.576528+0000","last_clean":"2024-03-07T07:20:09.576528+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:09.576528+0000","last_undegraded":"2024-03-07T07:20:09.576528+0000","last_fullsized":"2024-03-07T07:20:09.576528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017643+0000","last_change":"2024-03-07T07:20:08.023430+0000","last_active":"2024-03-07T07:20:09.017643+0000","last_peered":"2024-03-07T07:20:09.017643+0000","last_clean":"2024-03-07T07:20:09.017643+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:09.017643+0000","last_undegraded":"2024-03-07T07:20:09.017643+0000","last_fullsized":"2024-03-07T07:20:09.017643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:29.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576960+0000","last_change":"2024-03-07T07:20:08.033808+0000","last_active":"2024-03-07T07:20:09.576960+0000","last_peered":"2024-03-07T07:20:09.576960+0000","last_clean":"2024-03-07T07:20:09.576960+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:09.576960+0000","last_undegraded":"2024-03-07T07:20:09.576960+0000","last_fullsized":"2024-03-07T07:20:09.576960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015411+0000","last_change":"2024-03-07T07:20:08.030548+0000","last_active":"2024-03-07T07:20:09.015411+0000","last_peered":"2024-03-07T07:20:09.015411+0000","last_clean":"2024-03-07T07:20:09.015411+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:09.015411+0000","last_undegraded":"2024-03-07T07:20:09.015411+0000","last_fullsized":"2024-03-07T07:20:09.015411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018115+0000","last_change":"2024-03-07T07:20:07.025352+0000","last_active":"2024-03-07T07:20:09.018115+0000","last_peered":"2024-03-07T07:20:09.018115+0000","last_clean":"2024-03-07T07:20:09.018115+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:09.018115+0000","last_undegraded":"2024-03-07T07:20:09.018115+0000","last_fullsized":"2024-03-07T07:20:09.018115+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018004+0000","last_change":"2024-03-07T07:20:08.033169+0000","last_active":"2024-03-07T07:20:09.018004+0000","last_peered":"2024-03-07T07:20:09.018004+0000","last_clean":"2024-03-07T07:20:09.018004+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:09.018004+0000","last_undegraded":"2024-03-07T07:20:09.018004+0000","last_fullsized":"2024-03-07T07:20:09.018004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015349+0000","last_change":"2024-03-07T07:20:07.025175+0000","last_active":"2024-03-07T07:20:09.015349+0000","last_peered":"2024-03-07T07:20:09.015349+0000","last_clean":"2024-03-07T07:20:09.015349+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:09.015349+0000","last_undegraded":"2024-03-07T07:20:09.015349+0000","last_fullsized":"2024-03-07T07:20:09.015349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:23.151665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018559+0000","last_change":"2024-03-07T07:20:08.040222+0000","last_active":"2024-03-07T07:20:09.018559+0000","last_peered":"2024-03-07T07:20:09.018559+0000","last_clean":"2024-03-07T07:20:09.018559+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:09.018559+0000","last_undegraded":"2024-03-07T07:20:09.018559+0000","last_fullsized":"2024-03-07T07:20:09.018559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516719+0000","last_change":"2024-03-07T07:20:07.033158+0000","last_active":"2024-03-07T07:20:09.516719+0000","last_peered":"2024-03-07T07:20:09.516719+0000","last_clean":"2024-03-07T07:20:09.516719+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:09.516719+0000","last_undegraded":"2024-03-07T07:20:09.516719+0000","last_fullsized":"2024-03-07T07:20:09.516719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:30.672111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576436+0000","last_change":"2024-03-07T07:20:08.043374+0000","last_active":"2024-03-07T07:20:09.576436+0000","last_peered":"2024-03-07T07:20:09.576436+0000","last_clean":"2024-03-07T07:20:09.576436+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:09.576436+0000","last_undegraded":"2024-03-07T07:20:09.576436+0000","last_fullsized":"2024-03-07T07:20:09.576436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575854+0000","last_change":"2024-03-07T07:20:07.028373+0000","last_active":"2024-03-07T07:20:09.575854+0000","last_peered":"2024-03-07T07:20:09.575854+0000","last_clean":"2024-03-07T07:20:09.575854+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:09.575854+0000","last_undegraded":"2024-03-07T07:20:09.575854+0000","last_fullsized":"2024-03-07T07:20:09.575854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:51.229977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576660+0000","last_change":"2024-03-07T07:20:08.033882+0000","last_active":"2024-03-07T07:20:09.576660+0000","last_peered":"2024-03-07T07:20:09.576660+0000","last_clean":"2024-03-07T07:20:09.576660+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:09.576660+0000","last_undegraded":"2024-03-07T07:20:09.576660+0000","last_fullsized":"2024-03-07T07:20:09.576660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018255+0000","last_change":"2024-03-07T07:20:07.028711+0000","last_active":"2024-03-07T07:20:09.018255+0000","last_peered":"2024-03-07T07:20:09.018255+0000","last_clean":"2024-03-07T07:20:09.018255+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:09.018255+0000","last_undegraded":"2024-03-07T07:20:09.018255+0000","last_fullsized":"2024-03-07T07:20:09.018255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:46.879544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576378+0000","last_change":"2024-03-07T07:20:08.030098+0000","last_active":"2024-03-07T07:20:09.576378+0000","last_peered":"2024-03-07T07:20:09.576378+0000","last_clean":"2024-03-07T07:20:09.576378+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:09.576378+0000","last_undegraded":"2024-03-07T07:20:09.576378+0000","last_fullsized":"2024-03-07T07:20:09.576378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018003+0000","last_change":"2024-03-07T07:20:07.029834+0000","last_active":"2024-03-07T07:20:09.018003+0000","last_peered":"2024-03-07T07:20:09.018003+0000","last_clean":"2024-03-07T07:20:09.018003+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:09.018003+0000","last_undegraded":"2024-03-07T07:20:09.018003+0000","last_fullsized":"2024-03-07T07:20:09.018003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:55:49.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018248+0000","last_change":"2024-03-07T07:20:08.028053+0000","last_active":"2024-03-07T07:20:09.018248+0000","last_peered":"2024-03-07T07:20:09.018248+0000","last_clean":"2024-03-07T07:20:09.018248+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:09.018248+0000","last_undegraded":"2024-03-07T07:20:09.018248+0000","last_fullsized":"2024-03-07T07:20:09.018248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516477+0000","last_change":"2024-03-07T07:20:07.033411+0000","last_active":"2024-03-07T07:20:09.516477+0000","last_peered":"2024-03-07T07:20:09.516477+0000","last_clean":"2024-03-07T07:20:09.516477+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:09.516477+0000","last_undegraded":"2024-03-07T07:20:09.516477+0000","last_fullsized":"2024-03-07T07:20:09.516477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:15.200144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576038+0000","last_change":"2024-03-07T07:20:08.032643+0000","last_active":"2024-03-07T07:20:09.576038+0000","last_peered":"2024-03-07T07:20:09.576038+0000","last_clean":"2024-03-07T07:20:09.576038+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:09.576038+0000","last_undegraded":"2024-03-07T07:20:09.576038+0000","last_fullsized":"2024-03-07T07:20:09.576038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018364+0000","last_change":"2024-03-07T07:20:07.027022+0000","last_active":"2024-03-07T07:20:09.018364+0000","last_peered":"2024-03-07T07:20:09.018364+0000","last_clean":"2024-03-07T07:20:09.018364+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:09.018364+0000","last_undegraded":"2024-03-07T07:20:09.018364+0000","last_fullsized":"2024-03-07T07:20:09.018364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:35.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.028592+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516961+0000","last_change":"2024-03-07T07:20:07.030367+0000","last_active":"2024-03-07T07:20:09.516961+0000","last_peered":"2024-03-07T07:20:09.516961+0000","last_clean":"2024-03-07T07:20:09.516961+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:09.516961+0000","last_undegraded":"2024-03-07T07:20:09.516961+0000","last_fullsized":"2024-03-07T07:20:09.516961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:12.128114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516860+0000","last_change":"2024-03-07T07:20:08.034949+0000","last_active":"2024-03-07T07:20:09.516860+0000","last_peered":"2024-03-07T07:20:09.516860+0000","last_clean":"2024-03-07T07:20:09.516860+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:09.516860+0000","last_undegraded":"2024-03-07T07:20:09.516860+0000","last_fullsized":"2024-03-07T07:20:09.516860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015409+0000","last_change":"2024-03-07T07:20:07.030049+0000","last_active":"2024-03-07T07:20:09.015409+0000","last_peered":"2024-03-07T07:20:09.015409+0000","last_clean":"2024-03-07T07:20:09.015409+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:09.015409+0000","last_undegraded":"2024-03-07T07:20:09.015409+0000","last_fullsized":"2024-03-07T07:20:09.015409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516872+0000","last_change":"2024-03-07T07:20:08.030884+0000","last_active":"2024-03-07T07:20:09.516872+0000","last_peered":"2024-03-07T07:20:09.516872+0000","last_clean":"2024-03-07T07:20:09.516872+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:09.516872+0000","last_undegraded":"2024-03-07T07:20:09.516872+0000","last_fullsized":"2024-03-07T07:20:09.516872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:22.617306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577049+0000","last_change":"2024-03-07T07:20:07.028526+0000","last_active":"2024-03-07T07:20:09.577049+0000","last_peered":"2024-03-07T07:20:09.577049+0000","last_clean":"2024-03-07T07:20:09.577049+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:09.577049+0000","last_undegraded":"2024-03-07T07:20:09.577049+0000","last_fullsized":"2024-03-07T07:20:09.577049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576521+0000","last_change":"2024-03-07T07:20:08.038594+0000","last_active":"2024-03-07T07:20:09.576521+0000","last_peered":"2024-03-07T07:20:09.576521+0000","last_clean":"2024-03-07T07:20:09.576521+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:09.576521+0000","last_undegraded":"2024-03-07T07:20:09.576521+0000","last_fullsized":"2024-03-07T07:20:09.576521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'10","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:15.856571+0000","last_change":"2024-03-07T07:20:07.035824+0000","last_active":"2024-03-07T07:20:15.856571+0000","last_peered":"2024-03-07T07:20:15.856571+0000","last_clean":"2024-03-07T07:20:15.856571+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:15.856571+0000","last_undegraded":"2024-03-07T07:20:15.856571+0000","last_fullsized":"2024-03-07T07:20:15.856571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7145,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576168+0000","last_change":"2024-03-07T07:20:08.034611+0000","last_active":"2024-03-07T07:20:09.576168+0000","last_peered":"2024-03-07T07:20:09.576168+0000","last_clean":"2024-03-07T07:20:09.576168+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:09.576168+0000","last_undegraded":"2024-03-07T07:20:09.576168+0000","last_fullsized":"2024-03-07T07:20:09.576168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018274+0000","last_change":"2024-03-07T07:20:07.025705+0000","last_active":"2024-03-07T07:20:09.018274+0000","last_peered":"2024-03-07T07:20:09.018274+0000","last_clean":"2024-03-07T07:20:09.018274+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:09.018274+0000","last_undegraded":"2024-03-07T07:20:09.018274+0000","last_fullsized":"2024-03-07T07:20:09.018274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516759+0000","last_change":"2024-03-07T07:20:08.037145+0000","last_active":"2024-03-07T07:20:09.516759+0000","last_peered":"2024-03-07T07:20:09.516759+0000","last_clean":"2024-03-07T07:20:09.516759+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:09.516759+0000","last_undegraded":"2024-03-07T07:20:09.516759+0000","last_fullsized":"2024-03-07T07:20:09.516759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576262+0000","last_change":"2024-03-07T07:20:07.026780+0000","last_active":"2024-03-07T07:20:09.576262+0000","last_peered":"2024-03-07T07:20:09.576262+0000","last_clean":"2024-03-07T07:20:09.576262+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:09.576262+0000","last_undegraded":"2024-03-07T07:20:09.576262+0000","last_fullsized":"2024-03-07T07:20:09.576262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577126+0000","last_change":"2024-03-07T07:20:08.033869+0000","last_active":"2024-03-07T07:20:09.577126+0000","last_peered":"2024-03-07T07:20:09.577126+0000","last_clean":"2024-03-07T07:20:09.577126+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:09.577126+0000","last_undegraded":"2024-03-07T07:20:09.577126+0000","last_fullsized":"2024-03-07T07:20:09.577126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516453+0000","last_change":"2024-03-07T07:20:07.032636+0000","last_active":"2024-03-07T07:20:09.516453+0000","last_peered":"2024-03-07T07:20:09.516453+0000","last_clean":"2024-03-07T07:20:09.516453+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:09.516453+0000","last_undegraded":"2024-03-07T07:20:09.516453+0000","last_fullsized":"2024-03-07T07:20:09.516453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017781+0000","last_change":"2024-03-07T07:20:08.036782+0000","last_active":"2024-03-07T07:20:09.017781+0000","last_peered":"2024-03-07T07:20:09.017781+0000","last_clean":"2024-03-07T07:20:09.017781+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:09.017781+0000","last_undegraded":"2024-03-07T07:20:09.017781+0000","last_fullsized":"2024-03-07T07:20:09.017781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:19.197402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516567+0000","last_change":"2024-03-07T07:20:07.033230+0000","last_active":"2024-03-07T07:20:09.516567+0000","last_peered":"2024-03-07T07:20:09.516567+0000","last_clean":"2024-03-07T07:20:09.516567+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:09.516567+0000","last_undegraded":"2024-03-07T07:20:09.516567+0000","last_fullsized":"2024-03-07T07:20:09.516567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:42.777795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018625+0000","last_change":"2024-03-07T07:20:08.036934+0000","last_active":"2024-03-07T07:20:09.018625+0000","last_peered":"2024-03-07T07:20:09.018625+0000","last_clean":"2024-03-07T07:20:09.018625+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:09.018625+0000","last_undegraded":"2024-03-07T07:20:09.018625+0000","last_fullsized":"2024-03-07T07:20:09.018625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576460+0000","last_change":"2024-03-07T07:20:07.033018+0000","last_active":"2024-03-07T07:20:09.576460+0000","last_peered":"2024-03-07T07:20:09.576460+0000","last_clean":"2024-03-07T07:20:09.576460+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:09.576460+0000","last_undegraded":"2024-03-07T07:20:09.576460+0000","last_fullsized":"2024-03-07T07:20:09.576460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577155+0000","last_change":"2024-03-07T07:20:08.035720+0000","last_active":"2024-03-07T07:20:09.577155+0000","last_peered":"2024-03-07T07:20:09.577155+0000","last_clean":"2024-03-07T07:20:09.577155+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:09.577155+0000","last_undegraded":"2024-03-07T07:20:09.577155+0000","last_fullsized":"2024-03-07T07:20:09.577155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018650+0000","last_change":"2024-03-07T07:20:07.026560+0000","last_active":"2024-03-07T07:20:09.018650+0000","last_peered":"2024-03-07T07:20:09.018650+0000","last_clean":"2024-03-07T07:20:09.018650+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:09.018650+0000","last_undegraded":"2024-03-07T07:20:09.018650+0000","last_fullsized":"2024-03-07T07:20:09.018650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:52.614111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576186+0000","last_change":"2024-03-07T07:20:08.033852+0000","last_active":"2024-03-07T07:20:09.576186+0000","last_peered":"2024-03-07T07:20:09.576186+0000","last_clean":"2024-03-07T07:20:09.576186+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:09.576186+0000","last_undegraded":"2024-03-07T07:20:09.576186+0000","last_fullsized":"2024-03-07T07:20:09.576186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018510+0000","last_change":"2024-03-07T07:20:07.023668+0000","last_active":"2024-03-07T07:20:09.018510+0000","last_peered":"2024-03-07T07:20:09.018510+0000","last_clean":"2024-03-07T07:20:09.018510+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:09.018510+0000","last_undegraded":"2024-03-07T07:20:09.018510+0000","last_fullsized":"2024-03-07T07:20:09.018510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:07.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575815+0000","last_change":"2024-03-07T07:20:08.030524+0000","last_active":"2024-03-07T07:20:09.575815+0000","last_peered":"2024-03-07T07:20:09.575815+0000","last_clean":"2024-03-07T07:20:09.575815+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:09.575815+0000","last_undegraded":"2024-03-07T07:20:09.575815+0000","last_fullsized":"2024-03-07T07:20:09.575815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:18.342434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018550+0000","last_change":"2024-03-07T07:20:07.029892+0000","last_active":"2024-03-07T07:20:09.018550+0000","last_peered":"2024-03-07T07:20:09.018550+0000","last_clean":"2024-03-07T07:20:09.018550+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:09.018550+0000","last_undegraded":"2024-03-07T07:20:09.018550+0000","last_fullsized":"2024-03-07T07:20:09.018550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:24.735113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575612+0000","last_change":"2024-03-07T07:20:08.032134+0000","last_active":"2024-03-07T07:20:09.575612+0000","last_peered":"2024-03-07T07:20:09.575612+0000","last_clean":"2024-03-07T07:20:09.575612+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:09.575612+0000","last_undegraded":"2024-03-07T07:20:09.575612+0000","last_fullsized":"2024-03-07T07:20:09.575612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:48.726782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015375+0000","last_change":"2024-03-07T07:20:07.028153+0000","last_active":"2024-03-07T07:20:09.015375+0000","last_peered":"2024-03-07T07:20:09.015375+0000","last_clean":"2024-03-07T07:20:09.015375+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:09.015375+0000","last_undegraded":"2024-03-07T07:20:09.015375+0000","last_fullsized":"2024-03-07T07:20:09.015375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575806+0000","last_change":"2024-03-07T07:20:08.032594+0000","last_active":"2024-03-07T07:20:09.575806+0000","last_peered":"2024-03-07T07:20:09.575806+0000","last_clean":"2024-03-07T07:20:09.575806+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:09.575806+0000","last_undegraded":"2024-03-07T07:20:09.575806+0000","last_fullsized":"2024-03-07T07:20:09.575806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:15.552375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018303+0000","last_change":"2024-03-07T07:20:07.028455+0000","last_active":"2024-03-07T07:20:09.018303+0000","last_peered":"2024-03-07T07:20:09.018303+0000","last_clean":"2024-03-07T07:20:09.018303+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:09.018303+0000","last_undegraded":"2024-03-07T07:20:09.018303+0000","last_fullsized":"2024-03-07T07:20:09.018303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575541+0000","last_change":"2024-03-07T07:20:08.031299+0000","last_active":"2024-03-07T07:20:09.575541+0000","last_peered":"2024-03-07T07:20:09.575541+0000","last_clean":"2024-03-07T07:20:09.575541+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:09.575541+0000","last_undegraded":"2024-03-07T07:20:09.575541+0000","last_fullsized":"2024-03-07T07:20:09.575541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:24:00.520723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017852+0000","last_change":"2024-03-07T07:20:07.028369+0000","last_active":"2024-03-07T07:20:09.017852+0000","last_peered":"2024-03-07T07:20:09.017852+0000","last_clean":"2024-03-07T07:20:09.017852+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:09.017852+0000","last_undegraded":"2024-03-07T07:20:09.017852+0000","last_fullsized":"2024-03-07T07:20:09.017852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.033176+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576122+0000","last_change":"2024-03-07T07:20:07.029244+0000","last_active":"2024-03-07T07:20:09.576122+0000","last_peered":"2024-03-07T07:20:09.576122+0000","last_clean":"2024-03-07T07:20:09.576122+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:09.576122+0000","last_undegraded":"2024-03-07T07:20:09.576122+0000","last_fullsized":"2024-03-07T07:20:09.576122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575628+0000","last_change":"2024-03-07T07:20:08.026414+0000","last_active":"2024-03-07T07:20:09.575628+0000","last_peered":"2024-03-07T07:20:09.575628+0000","last_clean":"2024-03-07T07:20:09.575628+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:09.575628+0000","last_undegraded":"2024-03-07T07:20:09.575628+0000","last_fullsized":"2024-03-07T07:20:09.575628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:51.771656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576502+0000","last_change":"2024-03-07T07:20:07.027164+0000","last_active":"2024-03-07T07:20:09.576502+0000","last_peered":"2024-03-07T07:20:09.576502+0000","last_clean":"2024-03-07T07:20:09.576502+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:09.576502+0000","last_undegraded":"2024-03-07T07:20:09.576502+0000","last_fullsized":"2024-03-07T07:20:09.576502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576522+0000","last_change":"2024-03-07T07:20:08.035488+0000","last_active":"2024-03-07T07:20:09.576522+0000","last_peered":"2024-03-07T07:20:09.576522+0000","last_clean":"2024-03-07T07:20:09.576522+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:09.576522+0000","last_undegraded":"2024-03-07T07:20:09.576522+0000","last_fullsized":"2024-03-07T07:20:09.576522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:15.017567+0000","last_change":"2024-03-07T07:20:07.027534+0000","last_active":"2024-03-07T07:20:15.017567+0000","last_peered":"2024-03-07T07:20:15.017567+0000","last_clean":"2024-03-07T07:20:15.017567+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:15.017567+0000","last_undegraded":"2024-03-07T07:20:15.017567+0000","last_fullsized":"2024-03-07T07:20:15.017567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015303+0000","last_change":"2024-03-07T07:20:08.036269+0000","last_active":"2024-03-07T07:20:09.015303+0000","last_peered":"2024-03-07T07:20:09.015303+0000","last_clean":"2024-03-07T07:20:09.015303+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:09.015303+0000","last_undegraded":"2024-03-07T07:20:09.015303+0000","last_fullsized":"2024-03-07T07:20:09.015303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576245+0000","last_change":"2024-03-07T07:20:07.023056+0000","last_active":"2024-03-07T07:20:09.576245+0000","last_peered":"2024-03-07T07:20:09.576245+0000","last_clean":"2024-03-07T07:20:09.576245+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:09.576245+0000","last_undegraded":"2024-03-07T07:20:09.576245+0000","last_fullsized":"2024-03-07T07:20:09.576245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018197+0000","last_change":"2024-03-07T07:20:08.023363+0000","last_active":"2024-03-07T07:20:09.018197+0000","last_peered":"2024-03-07T07:20:09.018197+0000","last_clean":"2024-03-07T07:20:09.018197+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:09.018197+0000","last_undegraded":"2024-03-07T07:20:09.018197+0000","last_fullsized":"2024-03-07T07:20:09.018197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015478+0000","last_change":"2024-03-07T07:20:07.024131+0000","last_active":"2024-03-07T07:20:09.015478+0000","last_peered":"2024-03-07T07:20:09.015478+0000","last_clean":"2024-03-07T07:20:09.015478+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:09.015478+0000","last_undegraded":"2024-03-07T07:20:09.015478+0000","last_fullsized":"2024-03-07T07:20:09.015478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015390+0000","last_change":"2024-03-07T07:20:08.030007+0000","last_active":"2024-03-07T07:20:09.015390+0000","last_peered":"2024-03-07T07:20:09.015390+0000","last_clean":"2024-03-07T07:20:09.015390+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:09.015390+0000","last_undegraded":"2024-03-07T07:20:09.015390+0000","last_fullsized":"2024-03-07T07:20:09.015390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576311+0000","last_change":"2024-03-07T07:20:07.025544+0000","last_active":"2024-03-07T07:20:09.576311+0000","last_peered":"2024-03-07T07:20:09.576311+0000","last_clean":"2024-03-07T07:20:09.576311+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:09.576311+0000","last_undegraded":"2024-03-07T07:20:09.576311+0000","last_fullsized":"2024-03-07T07:20:09.576311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:18.193822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516464+0000","last_change":"2024-03-07T07:20:08.035274+0000","last_active":"2024-03-07T07:20:09.516464+0000","last_peered":"2024-03-07T07:20:09.516464+0000","last_clean":"2024-03-07T07:20:09.516464+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:09.516464+0000","last_undegraded":"2024-03-07T07:20:09.516464+0000","last_fullsized":"2024-03-07T07:20:09.516464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017946+0000","last_change":"2024-03-07T07:20:07.023385+0000","last_active":"2024-03-07T07:20:09.017946+0000","last_peered":"2024-03-07T07:20:09.017946+0000","last_clean":"2024-03-07T07:20:09.017946+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:09.017946+0000","last_undegraded":"2024-03-07T07:20:09.017946+0000","last_fullsized":"2024-03-07T07:20:09.017946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017838+0000","last_change":"2024-03-07T07:20:08.040036+0000","last_active":"2024-03-07T07:20:09.017838+0000","last_peered":"2024-03-07T07:20:09.017838+0000","last_clean":"2024-03-07T07:20:09.017838+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:09.017838+0000","last_undegraded":"2024-03-07T07:20:09.017838+0000","last_fullsized":"2024-03-07T07:20:09.017838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516475+0000","last_change":"2024-03-07T07:20:07.036824+0000","last_active":"2024-03-07T07:20:09.516475+0000","last_peered":"2024-03-07T07:20:09.516475+0000","last_clean":"2024-03-07T07:20:09.516475+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:09.516475+0000","last_undegraded":"2024-03-07T07:20:09.516475+0000","last_fullsized":"2024-03-07T07:20:09.516475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:20.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576659+0000","last_change":"2024-03-07T07:20:08.034942+0000","last_active":"2024-03-07T07:20:09.576659+0000","last_peered":"2024-03-07T07:20:09.576659+0000","last_clean":"2024-03-07T07:20:09.576659+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:09.576659+0000","last_undegraded":"2024-03-07T07:20:09.576659+0000","last_fullsized":"2024-03-07T07:20:09.576659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017654+0000","last_change":"2024-03-07T07:20:07.023606+0000","last_active":"2024-03-07T07:20:09.017654+0000","last_peered":"2024-03-07T07:20:09.017654+0000","last_clean":"2024-03-07T07:20:09.017654+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:09.017654+0000","last_undegraded":"2024-03-07T07:20:09.017654+0000","last_fullsized":"2024-03-07T07:20:09.017654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":7943,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":15886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:20:20.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T07:20:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.411+0000 7f49679b1640 1 Processor -- start 2024-03-07T07:20:20.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.411+0000 7f49679b1640 1 -- start start 2024-03-07T07:20:20.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.411+0000 7f49679b1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49600d1860 0x7f496006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f49679b1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f49600b7390 0x7f496006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f49679b1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49600d0aa0 con 0x7f4960145bf0 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f49679b1640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600d13d0 con 0x7f49600d1860 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f49679b1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600d0d80 con 0x7f49600b7390 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f4964f4c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49600d1860 0x7f496006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f4964f4c640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f49600d1860 0x7f496006b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60560/0 (socket says 172.21.15.64:60560) 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f4964f4c640 1 -- 172.21.15.64:0/3796915560 learned_addr learned my addr 172.21.15.64:0/3796915560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:20.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f4964f4c640 1 -- 172.21.15.64:0/3796915560 >> v2:172.21.15.120:3300/0 conn(0x7f49600b7390 msgr2=0x7f496006ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f4964f4c640 1 --2- 172.21.15.64:0/3796915560 >> v2:172.21.15.120:3300/0 conn(0x7f49600b7390 0x7f496006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f4964f4c640 1 -- 172.21.15.64:0/3796915560 >> v1:172.21.15.120:6789/0 conn(0x7f4960145bf0 legacy=0x7f49600cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.412+0000 7f4964f4c640 1 -- 172.21.15.64:0/3796915560 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600d1060 con 0x7f49600d1860 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f4964f4c640 1 --2- 172.21.15.64:0/3796915560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600d1860 0x7f496006b510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f49600b6ae0 tx=0x7f495002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8262d43ac32d99b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f4957fff640 1 -- 172.21.15.64:0/3796915560 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950035020 con 0x7f49600d1860 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f4957fff640 1 -- 172.21.15.64:0/3796915560 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4950031870 con 0x7f49600d1860 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f4957fff640 1 -- 172.21.15.64:0/3796915560 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950031c40 con 0x7f49600d1860 2024-03-07T07:20:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f49679b1640 1 -- 172.21.15.64:0/3796915560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600d1860 msgr2=0x7f496006b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f49679b1640 1 --2- 172.21.15.64:0/3796915560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600d1860 0x7f496006b510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f49600b6ae0 tx=0x7f495002b910 comp rx=0 tx=0).stop 2024-03-07T07:20:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f49679b1640 1 -- 172.21.15.64:0/3796915560 shutdown_connections 2024-03-07T07:20:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f49679b1640 1 --2- 172.21.15.64:0/3796915560 >> v2:172.21.15.120:3300/0 conn(0x7f49600b7390 0x7f496006ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f49679b1640 1 --2- 172.21.15.64:0/3796915560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600d1860 0x7f496006b510 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.413+0000 7f49679b1640 1 -- 172.21.15.64:0/3796915560 >> 172.21.15.64:0/3796915560 conn(0x7f496006d4c0 msgr2=0x7f4960066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.414+0000 7f49679b1640 1 -- 172.21.15.64:0/3796915560 shutdown_connections 2024-03-07T07:20:20.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.414+0000 7f49679b1640 1 -- 172.21.15.64:0/3796915560 wait complete. 2024-03-07T07:20:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 Processor -- start 2024-03-07T07:20:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 -- start start 2024-03-07T07:20:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600b7390 0x7f49601f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49600d1860 0x7f496014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4960145bf0 0x7f49600c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600d0840 con 0x7f49600b7390 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600d06c0 con 0x7f4960145bf0 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f49679b1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49600b5160 con 0x7f49600d1860 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f496574d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600b7390 0x7f49601f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f496574d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600b7390 0x7f49601f2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33226/0 (socket says 172.21.15.64:33226) 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.415+0000 7f496574d640 1 -- 172.21.15.64:0/3756972127 learned_addr learned my addr 172.21.15.64:0/3756972127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f496574d640 1 -- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49600d1860 msgr2=0x7f496014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f496574d640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49600d1860 0x7f496014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f496574d640 1 -- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4960145bf0 msgr2=0x7f49600c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f4965f4e640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4960145bf0 0x7f49600c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f496574d640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4960145bf0 0x7f49600c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f496574d640 1 -- 172.21.15.64:0/3756972127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496014f070 con 0x7f49600b7390 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f496574d640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600b7390 0x7f49601f2280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b3e0 tx=0x7f495c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f4955ffb640 1 -- 172.21.15.64:0/3756972127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c015020 con 0x7f49600b7390 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f4955ffb640 1 -- 172.21.15.64:0/3756972127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495c011850 con 0x7f49600b7390 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f4955ffb640 1 -- 172.21.15.64:0/3756972127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c011c40 con 0x7f49600b7390 2024-03-07T07:20:20.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496015dc90 con 0x7f49600b7390 2024-03-07T07:20:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.416+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496015e250 con 0x7f49600b7390 2024-03-07T07:20:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.418+0000 7f4955ffb640 1 -- 172.21.15.64:0/3756972127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f495c018020 con 0x7f49600b7390 2024-03-07T07:20:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.418+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4960062c90 con 0x7f49600b7390 2024-03-07T07:20:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.418+0000 7f4955ffb640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f494406b4d0 0x7f494406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.418+0000 7f4955ffb640 1 -- 172.21.15.64:0/3756972127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f495c08a090 con 0x7f49600b7390 2024-03-07T07:20:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.418+0000 7f4964f4c640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f494406b4d0 0x7f494406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:20.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.419+0000 7f4964f4c640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f494406b4d0 0x7f494406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4950036000 tx=0x7f4950030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:20.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.422+0000 7f4955ffb640 1 -- 172.21.15.64:0/3756972127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f495c01f020 con 0x7f49600b7390 2024-03-07T07:20:20.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.604+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49600b4b60 con 0x7f49600b7390 2024-03-07T07:20:20.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.605+0000 7f4955ffb640 1 -- 172.21.15.64:0/3756972127 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f495c0172c0 con 0x7f49600b7390 2024-03-07T07:20:20.605 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:20.605 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"fsid":"6e3a2ac9-a58d-43cb-a91c-68e2cc4ac8b3","created":"2024-03-07T07:18:51.913140+0000","modified":"2024-03-07T07:20:09.012362+0000","last_up_change":"2024-03-07T07:19:03.245404+0000","last_in_change":"2024-03-07T07:18:54.154592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T07:19:04.222822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T07:20:05.640176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T07:20:06.344694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6778ad92-ea90-4f4e-800a-a1934ea4f756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6801","nonce":382233877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6803","nonce":382233877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6807","nonce":382233877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":382233877},{"type":"v1","addr":"172.21.15.64:6805","nonce":382233877}]},"public_addr":"172.21.15.64:6801/382233877","cluster_addr":"172.21.15.64:6803/382233877","heartbeat_back_addr":"172.21.15.64:6807/382233877","heartbeat_front_addr":"172.21.15.64:6805/382233877","state":["exists","up"]},{"osd":1,"uuid":"b56d13e1-6c08-4768-8b35-cbaa4ca12a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6809","nonce":372208353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6811","nonce":372208353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6815","nonce":372208353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":372208353},{"type":"v1","addr":"172.21.15.64:6813","nonce":372208353}]},"public_addr":"172.21.15.64:6809/372208353","cluster_addr":"172.21.15.64:6811/372208353","heartbeat_back_addr":"172.21.15.64:6815/372208353","heartbeat_front_addr":"172.21.15.64:6813/372208353","state":["exists","up"]},{"osd":2,"uuid":"7fe794a2-99e7-4e41-aacc-573fee75b2ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6817","nonce":3037744413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6819","nonce":3037744413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6823","nonce":3037744413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":3037744413},{"type":"v1","addr":"172.21.15.64:6821","nonce":3037744413}]},"public_addr":"172.21.15.64:6817/3037744413","cluster_addr":"172.21.15.64:6819/3037744413","heartbeat_back_addr":"172.21.15.64:6823/3037744413","heartbeat_front_addr":"172.21.15.64:6821/3037744413","state":["exists","up"]},{"osd":3,"uuid":"49430b37-e1cf-41c4-89d9-d463a63b8412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6825","nonce":2270479097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6827","nonce":2270479097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6831","nonce":2270479097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2270479097},{"type":"v1","addr":"172.21.15.64:6829","nonce":2270479097}]},"public_addr":"172.21.15.64:6825/2270479097","cluster_addr":"172.21.15.64:6827/2270479097","heartbeat_back_addr":"172.21.15.64:6831/2270479097","heartbeat_front_addr":"172.21.15.64:6829/2270479097","state":["exists","up"]},{"osd":4,"uuid":"3df056f1-81c0-4587-9b2b-66de4f62d0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6809","nonce":505090945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6811","nonce":505090945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6815","nonce":505090945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":505090945},{"type":"v1","addr":"172.21.15.120:6813","nonce":505090945}]},"public_addr":"172.21.15.120:6809/505090945","cluster_addr":"172.21.15.120:6811/505090945","heartbeat_back_addr":"172.21.15.120:6815/505090945","heartbeat_front_addr":"172.21.15.120:6813/505090945","state":["exists","up"]},{"osd":5,"uuid":"87eae5ba-3309-4484-ad81-48d1bcbe2dde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6801","nonce":349750424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6803","nonce":349750424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6807","nonce":349750424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":349750424},{"type":"v1","addr":"172.21.15.120:6805","nonce":349750424}]},"public_addr":"172.21.15.120:6801/349750424","cluster_addr":"172.21.15.120:6803/349750424","heartbeat_back_addr":"172.21.15.120:6807/349750424","heartbeat_front_addr":"172.21.15.120:6805/349750424","state":["exists","up"]},{"osd":6,"uuid":"da015db4-af8c-4064-8f8a-c6fce89f0476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6825","nonce":2639829384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6827","nonce":2639829384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6831","nonce":2639829384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2639829384},{"type":"v1","addr":"172.21.15.120:6829","nonce":2639829384}]},"public_addr":"172.21.15.120:6825/2639829384","cluster_addr":"172.21.15.120:6827/2639829384","heartbeat_back_addr":"172.21.15.120:6831/2639829384","heartbeat_front_addr":"172.21.15.120:6829/2639829384","state":["exists","up"]},{"osd":7,"uuid":"219c980c-69d7-4196-baf2-944263ffa8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6817","nonce":1578261849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6819","nonce":1578261849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6823","nonce":1578261849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1578261849},{"type":"v1","addr":"172.21.15.120:6821","nonce":1578261849}]},"public_addr":"172.21.15.120:6817/1578261849","cluster_addr":"172.21.15.120:6819/1578261849","heartbeat_back_addr":"172.21.15.120:6823/1578261849","heartbeat_front_addr":"172.21.15.120:6821/1578261849","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.287641+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.288252+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.345307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.293993+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.334256+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.279461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.399113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T07:18:59.342178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T07:20:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f494406b4d0 msgr2=0x7f494406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f494406b4d0 0x7f494406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4950036000 tx=0x7f4950030000 comp rx=0 tx=0).stop 2024-03-07T07:20:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600b7390 msgr2=0x7f49601f2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:20.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600b7390 0x7f49601f2280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b3e0 tx=0x7f495c00b8a0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 shutdown_connections 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f494406b4d0 0x7f494406d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4960145bf0 0x7f49600c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49600d1860 0x7f496014eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 --2- 172.21.15.64:0/3756972127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49600b7390 0x7f49601f2280 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 >> 172.21.15.64:0/3756972127 conn(0x7f496006d4c0 msgr2=0x7f49600d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 shutdown_connections 2024-03-07T07:20:20.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:20.607+0000 7f49679b1640 1 -- 172.21.15.64:0/3756972127 wait complete. 2024-03-07T07:20:21.619 INFO:tasks.ceph:Scrubbing osd.0 2024-03-07T07:20:21.620 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:21.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.728+0000 7f88aeb89640 1 Processor -- start 2024-03-07T07:20:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.728+0000 7f88aeb89640 1 -- start start 2024-03-07T07:20:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88aeb89640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f88a80b4e70 0x7f88a806b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88aeb89640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88a806ba50 0x7f88a80d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88aeb89640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a80b5490 con 0x7f88a8145920 2024-03-07T07:20:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88aeb89640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80b60a0 con 0x7f88a806ba50 2024-03-07T07:20:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88aeb89640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80b5770 con 0x7f88a80b4e70 2024-03-07T07:20:21.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88ad126640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f88a806ba50 0x7f88a80d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88ac925640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f88a8145920 0x7f88a80b6c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52426/0 (socket says 172.21.15.64:52426) 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.729+0000 7f88ac925640 1 -- 172.21.15.64:0/11727319 learned_addr learned my addr 172.21.15.64:0/11727319 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.730+0000 7f88a7fff640 1 --2- 172.21.15.64:0/11727319 >> v2:172.21.15.120:3300/0 conn(0x7f88a80b4e70 0x7f88a806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.730+0000 7f88ad126640 1 -- 172.21.15.64:0/11727319 >> v2:172.21.15.120:3300/0 conn(0x7f88a80b4e70 msgr2=0x7f88a806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.730+0000 7f88ad126640 1 --2- 172.21.15.64:0/11727319 >> v2:172.21.15.120:3300/0 conn(0x7f88a80b4e70 0x7f88a806b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.730+0000 7f88ad126640 1 -- 172.21.15.64:0/11727319 >> v1:172.21.15.64:6789/0 conn(0x7f88a8145920 legacy=0x7f88a80b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.730+0000 7f88a77fe640 1 -- 172.21.15.64:0/11727319 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744685973 0 0) 0x7f88a80b5490 con 0x7f88a8145920 2024-03-07T07:20:21.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.730+0000 7f88ad126640 1 -- 172.21.15.64:0/11727319 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80b5a50 con 0x7f88a806ba50 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.730+0000 7f88a7fff640 1 --2- 172.21.15.64:0/11727319 >> v2:172.21.15.120:3300/0 conn(0x7f88a80b4e70 0x7f88a806b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88ad126640 1 --2- 172.21.15.64:0/11727319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a806ba50 0x7f88a80d04d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f88a80b6890 tx=0x7f889802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e099d321a39af28 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88a77fe640 1 -- 172.21.15.64:0/11727319 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88980315a0 con 0x7f88a806ba50 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88a77fe640 1 -- 172.21.15.64:0/11727319 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8898037070 con 0x7f88a806ba50 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88a77fe640 1 -- 172.21.15.64:0/11727319 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898031960 con 0x7f88a806ba50 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 -- 172.21.15.64:0/11727319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a806ba50 msgr2=0x7f88a80d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 --2- 172.21.15.64:0/11727319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a806ba50 0x7f88a80d04d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f88a80b6890 tx=0x7f889802b840 comp rx=0 tx=0).stop 2024-03-07T07:20:21.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 -- 172.21.15.64:0/11727319 shutdown_connections 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 --2- 172.21.15.64:0/11727319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a806ba50 0x7f88a80d04d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 --2- 172.21.15.64:0/11727319 >> v2:172.21.15.120:3300/0 conn(0x7f88a80b4e70 0x7f88a806b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 -- 172.21.15.64:0/11727319 >> 172.21.15.64:0/11727319 conn(0x7f88a8066e50 msgr2=0x7f88a8067260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 -- 172.21.15.64:0/11727319 shutdown_connections 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.731+0000 7f88aeb89640 1 -- 172.21.15.64:0/11727319 wait complete. 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 Processor -- start 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 -- start start 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a806ba50 0x7f88a813b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80b4e70 0x7f88a813e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8145920 0x7f88a81409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80d0a10 con 0x7f88a80b4e70 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a8145d00 con 0x7f88a8145920 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.733+0000 7f88aeb89640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80d0b90 con 0x7f88a806ba50 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80b4e70 0x7f88a813e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80b4e70 0x7f88a813e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33250/0 (socket says 172.21.15.64:33250) 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 -- 172.21.15.64:0/4071357592 learned_addr learned my addr 172.21.15.64:0/4071357592 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:21.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88ad126640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8145920 0x7f88a81409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88ac925640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a806ba50 0x7f88a813b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 -- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a806ba50 msgr2=0x7f88a813b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a806ba50 0x7f88a813b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 -- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8145920 msgr2=0x7f88a81409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88a8145920 0x7f88a81409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 -- 172.21.15.64:0/4071357592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a813f2a0 con 0x7f88a80b4e70 2024-03-07T07:20:21.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88ac925640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88a806ba50 0x7f88a813b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a7fff640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80b4e70 0x7f88a813e9a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f889c00c7f0 tx=0x7f889c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c015070 con 0x7f88a80b4e70 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c007a30 con 0x7f88a80b4e70 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a813f580 con 0x7f88a80b4e70 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c007e00 con 0x7f88a80b4e70 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.734+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a813eee0 con 0x7f88a80b4e70 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.735+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88a813f7c0 con 0x7f88a80b4e70 2024-03-07T07:20:21.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f889c018070 con 0x7f88a80b4e70 2024-03-07T07:20:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88a57fa640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f889c08a1b0 con 0x7f88a80b4e70 2024-03-07T07:20:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88a57fa640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7f8894073670 0x7f8894075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 --> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8894076140 con 0x7f8894073670 2024-03-07T07:20:21.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c091050 con 0x7f88a80b4e70 2024-03-07T07:20:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88ac925640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.736+0000 7f88ad126640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7f8894073670 0x7f8894075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.737+0000 7f88ad126640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7f8894073670 0x7f8894075a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.737+0000 7f88ac925640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f88a0007fd0 tx=0x7f88a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:21.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.737+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== osd.0 v2:172.21.15.64:6800/382233877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8894076140 con 0x7f8894073670 2024-03-07T07:20:21.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 --> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f88a8062c90 con 0x7f8894073670 2024-03-07T07:20:21.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88a57fa640 1 -- 172.21.15.64:0/4071357592 <== osd.0 v2:172.21.15.64:6800/382233877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f88a8062c90 con 0x7f8894073670 2024-03-07T07:20:21.751 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:21.751 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:21.751 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:21.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7f8894073670 msgr2=0x7f8894075a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:6800/382233877,v1:172.21.15.64:6801/382233877] conn(0x7f8894073670 0x7f8894075a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 msgr2=0x7f889406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f889406b4d0 0x7f889406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f88a0007fd0 tx=0x7f88a0009040 comp rx=0 tx=0).stop 2024-03-07T07:20:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80b4e70 msgr2=0x7f88a813e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 --2- 172.21.15.64:0/4071357592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88a80b4e70 0x7f88a813e9a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f889c00c7f0 tx=0x7f889c00cbf0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88ad126640 1 -- 172.21.15.64:0/4071357592 reap_dead start 2024-03-07T07:20:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 shutdown_connections 2024-03-07T07:20:21.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 >> 172.21.15.64:0/4071357592 conn(0x7f88a8066e50 msgr2=0x7f88a80b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:21.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 shutdown_connections 2024-03-07T07:20:21.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.751+0000 7f88aeb89640 1 -- 172.21.15.64:0/4071357592 wait complete. 2024-03-07T07:20:21.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-07T07:20:21.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.864+0000 7fa8764dd640 1 Processor -- start 2024-03-07T07:20:21.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.864+0000 7fa8764dd640 1 -- start start 2024-03-07T07:20:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.864+0000 7fa8764dd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa870147270 0x7fa87006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.864+0000 7fa8764dd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa870147810 0x7fa87006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.864+0000 7fa8764dd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8700d1e70 con 0x7fa8700d2c10 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.864+0000 7fa8764dd640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa870147650 con 0x7fa870147810 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.864+0000 7fa8764dd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700b7520 con 0x7fa870147270 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa874a7a640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fa870147810 0x7fa87006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa86f7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fa8700d2c10 0x7fa870162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52440/0 (socket says 172.21.15.64:52440) 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa86f7fe640 1 -- 172.21.15.64:0/1940720634 learned_addr learned my addr 172.21.15.64:0/1940720634 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa86ffff640 1 --2- 172.21.15.64:0/1940720634 >> v2:172.21.15.120:3300/0 conn(0x7fa870147270 0x7fa87006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa874a7a640 1 -- 172.21.15.64:0/1940720634 >> v2:172.21.15.120:3300/0 conn(0x7fa870147270 msgr2=0x7fa87006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa874a7a640 1 --2- 172.21.15.64:0/1940720634 >> v2:172.21.15.120:3300/0 conn(0x7fa870147270 0x7fa87006b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa874a7a640 1 -- 172.21.15.64:0/1940720634 >> v1:172.21.15.64:6789/0 conn(0x7fa8700d2c10 legacy=0x7fa870162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa874a7a640 1 -- 172.21.15.64:0/1940720634 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700d2130 con 0x7fa870147810 2024-03-07T07:20:21.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.865+0000 7fa86effd640 1 -- 172.21.15.64:0/1940720634 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963017905 0 0) 0x7fa8700d1e70 con 0x7fa8700d2c10 2024-03-07T07:20:21.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.866+0000 7fa874a7a640 1 --2- 172.21.15.64:0/1940720634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa870147810 0x7fa87006ba50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa87006dd60 tx=0x7fa86802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb64d715c5393bc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:21.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.866+0000 7fa86effd640 1 -- 172.21.15.64:0/1940720634 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868039070 con 0x7fa870147810 2024-03-07T07:20:21.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.866+0000 7fa86effd640 1 -- 172.21.15.64:0/1940720634 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86802cc80 con 0x7fa870147810 2024-03-07T07:20:21.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.866+0000 7fa86effd640 1 -- 172.21.15.64:0/1940720634 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868034660 con 0x7fa870147810 2024-03-07T07:20:21.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 -- 172.21.15.64:0/1940720634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa870147810 msgr2=0x7fa87006ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 --2- 172.21.15.64:0/1940720634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa870147810 0x7fa87006ba50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa87006dd60 tx=0x7fa86802bed0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 -- 172.21.15.64:0/1940720634 shutdown_connections 2024-03-07T07:20:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 --2- 172.21.15.64:0/1940720634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa870147810 0x7fa87006ba50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 --2- 172.21.15.64:0/1940720634 >> v2:172.21.15.120:3300/0 conn(0x7fa870147270 0x7fa87006b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 -- 172.21.15.64:0/1940720634 >> 172.21.15.64:0/1940720634 conn(0x7fa87006d4c0 msgr2=0x7fa870066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 -- 172.21.15.64:0/1940720634 shutdown_connections 2024-03-07T07:20:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.867+0000 7fa8764dd640 1 -- 172.21.15.64:0/1940720634 wait complete. 2024-03-07T07:20:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.868+0000 7fa8764dd640 1 Processor -- start 2024-03-07T07:20:21.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.868+0000 7fa8764dd640 1 -- start start 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa8764dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8700d2c10 0x7fa87015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa8764dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa870147270 0x7fa870141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa8764dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa870147810 0x7fa8700d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa8764dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700b71d0 con 0x7fa8700d2c10 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa8764dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700d26f0 con 0x7fa870147810 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa8764dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa870147d60 con 0x7fa870147270 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8700d2c10 0x7fa87015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8700d2c10 0x7fa87015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33264/0 (socket says 172.21.15.64:33264) 2024-03-07T07:20:21.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 -- 172.21.15.64:0/3601981182 learned_addr learned my addr 172.21.15.64:0/3601981182 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa874a7a640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa870147810 0x7fa8700d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86f7fe640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa870147270 0x7fa870141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 -- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa870147270 msgr2=0x7fa870141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa870147270 0x7fa870141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 -- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa870147810 msgr2=0x7fa8700d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa870147810 0x7fa8700d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 -- 172.21.15.64:0/3601981182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700cc570 con 0x7fa8700d2c10 2024-03-07T07:20:21.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa874a7a640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa870147810 0x7fa8700d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.869+0000 7fa86ffff640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8700d2c10 0x7fa87015fd60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa86000d830 tx=0x7fa86000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.870+0000 7fa86cff9640 1 -- 172.21.15.64:0/3601981182 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000b6c0 con 0x7fa8700d2c10 2024-03-07T07:20:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.870+0000 7fa86cff9640 1 -- 172.21.15.64:0/3601981182 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860019070 con 0x7fa8700d2c10 2024-03-07T07:20:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.870+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8700cf410 con 0x7fa8700d2c10 2024-03-07T07:20:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.870+0000 7fa86cff9640 1 -- 172.21.15.64:0/3601981182 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000ba80 con 0x7fa8700d2c10 2024-03-07T07:20:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.870+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8700cc840 con 0x7fa8700d2c10 2024-03-07T07:20:21.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.871+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa870062c90 con 0x7fa8700d2c10 2024-03-07T07:20:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.871+0000 7fa86cff9640 1 -- 172.21.15.64:0/3601981182 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa86001e460 con 0x7fa8700d2c10 2024-03-07T07:20:21.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.872+0000 7fa86cff9640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.872+0000 7fa86cff9640 1 -- 172.21.15.64:0/3601981182 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa86008a8a0 con 0x7fa8700d2c10 2024-03-07T07:20:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.872+0000 7fa86f7fe640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.872+0000 7fa86f7fe640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa85c06b4d0 0x7fa85c06d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8701f7240 tx=0x7fa864007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:21.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:21.876+0000 7fa86cff9640 1 -- 172.21.15.64:0/3601981182 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa8600597b0 con 0x7fa8700d2c10 2024-03-07T07:20:22.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.056+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8701602a0 con 0x7fa85c06b4d0 2024-03-07T07:20:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.057+0000 7fa86cff9640 1 -- 172.21.15.64:0/3601981182 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa8701602a0 con 0x7fa85c06b4d0 2024-03-07T07:20:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 0 to deep-scrub 2024-03-07T07:20:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.058+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa85c06b4d0 msgr2=0x7fa85c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.058+0000 7fa8764dd640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa85c06b4d0 0x7fa85c06d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8701f7240 tx=0x7fa864007b50 comp rx=0 tx=0).stop 2024-03-07T07:20:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.058+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8700d2c10 msgr2=0x7fa87015fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.058+0000 7fa8764dd640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8700d2c10 0x7fa87015fd60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa86000d830 tx=0x7fa86000dcf0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 shutdown_connections 2024-03-07T07:20:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa870147810 0x7fa8700d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa870147270 0x7fa870141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 --2- 172.21.15.64:0/3601981182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8700d2c10 0x7fa87015fd60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 >> 172.21.15.64:0/3601981182 conn(0x7fa87006d4c0 msgr2=0x7fa87006b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 shutdown_connections 2024-03-07T07:20:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.059+0000 7fa8764dd640 1 -- 172.21.15.64:0/3601981182 wait complete. 2024-03-07T07:20:22.070 INFO:tasks.ceph:Scrubbing osd.1 2024-03-07T07:20:22.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.201+0000 7feb6851d640 1 Processor -- start 2024-03-07T07:20:22.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.201+0000 7feb6851d640 1 -- start start 2024-03-07T07:20:22.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb6851d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb6015f410 0x7feb600b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb6851d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7feb600b4040 0x7feb60142060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb6851d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6013f800 con 0x7feb6015d6b0 2024-03-07T07:20:22.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb6851d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6013f370 con 0x7feb600b4040 2024-03-07T07:20:22.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb6851d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb600b44a0 con 0x7feb6015f410 2024-03-07T07:20:22.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb662b9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb6015f410 0x7feb600b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb66aba640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7feb600b4040 0x7feb60142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb65ab8640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7feb6015d6b0 0x7feb60141950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:51266/0 (socket says 172.21.15.64:51266) 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.202+0000 7feb65ab8640 1 -- 172.21.15.64:0/2832016829 learned_addr learned my addr 172.21.15.64:0/2832016829 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.203+0000 7feb66aba640 1 -- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb6015f410 msgr2=0x7feb600b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.203+0000 7feb66aba640 1 --2- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb6015f410 0x7feb600b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.203+0000 7feb66aba640 1 -- 172.21.15.64:0/2832016829 >> v1:172.21.15.120:6789/0 conn(0x7feb6015d6b0 legacy=0x7feb60141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.203+0000 7feb66aba640 1 -- 172.21.15.64:0/2832016829 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6013fae0 con 0x7feb600b4040 2024-03-07T07:20:22.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.203+0000 7feb662b9640 1 --2- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb6015f410 0x7feb600b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:22.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb66aba640 1 --2- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb600b4040 0x7feb60142060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb58009d90 tx=0x7feb5802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf13b502516cc17 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb652b7640 1 -- 172.21.15.64:0/2832016829 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58039070 con 0x7feb600b4040 2024-03-07T07:20:22.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb652b7640 1 -- 172.21.15.64:0/2832016829 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5802cbc0 con 0x7feb600b4040 2024-03-07T07:20:22.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb652b7640 1 -- 172.21.15.64:0/2832016829 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58034440 con 0x7feb600b4040 2024-03-07T07:20:22.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 -- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb600b4040 msgr2=0x7feb60142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 --2- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb600b4040 0x7feb60142060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb58009d90 tx=0x7feb5802be10 comp rx=0 tx=0).stop 2024-03-07T07:20:22.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 -- 172.21.15.64:0/2832016829 shutdown_connections 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 --2- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb600b4040 0x7feb60142060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 --2- 172.21.15.64:0/2832016829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb6015f410 0x7feb600b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 -- 172.21.15.64:0/2832016829 >> 172.21.15.64:0/2832016829 conn(0x7feb6006d970 msgr2=0x7feb60066eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 -- 172.21.15.64:0/2832016829 shutdown_connections 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.204+0000 7feb6851d640 1 -- 172.21.15.64:0/2832016829 wait complete. 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.205+0000 7feb6851d640 1 Processor -- start 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb6851d640 1 -- start start 2024-03-07T07:20:22.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb6851d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb600b4040 0x7feb6013aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb6851d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6015d6b0 0x7feb600d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb6851d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb6015f410 0x7feb601600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb6851d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6015f970 con 0x7feb6015d6b0 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb6851d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6015f7f0 con 0x7feb6015f410 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb6851d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb601405c0 con 0x7feb600b4040 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb65ab8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6015d6b0 0x7feb600d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb65ab8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6015d6b0 0x7feb600d1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33266/0 (socket says 172.21.15.64:33266) 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb65ab8640 1 -- 172.21.15.64:0/583010370 learned_addr learned my addr 172.21.15.64:0/583010370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:22.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb662b9640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb600b4040 0x7feb6013aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.206+0000 7feb66aba640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb6015f410 0x7feb601600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb65ab8640 1 -- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb600b4040 msgr2=0x7feb6013aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb65ab8640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feb600b4040 0x7feb6013aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb65ab8640 1 -- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb6015f410 msgr2=0x7feb601600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb65ab8640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb6015f410 0x7feb601600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb65ab8640 1 -- 172.21.15.64:0/583010370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600721c0 con 0x7feb6015d6b0 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb65ab8640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6015d6b0 0x7feb600d1d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c430 tx=0x7feb5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50017070 con 0x7feb6015d6b0 2024-03-07T07:20:22.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb50005ce0 con 0x7feb6015d6b0 2024-03-07T07:20:22.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb600724a0 con 0x7feb6015d6b0 2024-03-07T07:20:22.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb500072f0 con 0x7feb6015d6b0 2024-03-07T07:20:22.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.207+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb60072a60 con 0x7feb6015d6b0 2024-03-07T07:20:22.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.208+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb60062c90 con 0x7feb6015d6b0 2024-03-07T07:20:22.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7feb5001a070 con 0x7feb6015d6b0 2024-03-07T07:20:22.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb56ffd640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7feb4406b4d0 0x7feb4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb50089600 con 0x7feb6015d6b0 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb56ffd640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7feb44073670 0x7feb44075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 --> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb50010e50 con 0x7feb44073670 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb662b9640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7feb4406b4d0 0x7feb4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb50090050 con 0x7feb6015d6b0 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.209+0000 7feb66aba640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7feb44073670 0x7feb44075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.210+0000 7feb662b9640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7feb4406b4d0 0x7feb4406d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009560 tx=0x7feb5c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.210+0000 7feb66aba640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7feb44073670 0x7feb44075a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.211+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== osd.1 v2:172.21.15.64:6808/372208353 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7feb50010e50 con 0x7feb44073670 2024-03-07T07:20:22.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.225+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 --> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feb6006c0a0 con 0x7feb44073670 2024-03-07T07:20:22.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb56ffd640 1 -- 172.21.15.64:0/583010370 <== osd.1 v2:172.21.15.64:6808/372208353 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7feb6006c0a0 con 0x7feb44073670 2024-03-07T07:20:22.226 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:22.226 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:22.227 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:22.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7feb44073670 msgr2=0x7feb44075a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:6808/372208353,v1:172.21.15.64:6809/372208353] conn(0x7feb44073670 0x7feb44075a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7feb4406b4d0 msgr2=0x7feb4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7feb4406b4d0 0x7feb4406d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009560 tx=0x7feb5c009190 comp rx=0 tx=0).stop 2024-03-07T07:20:22.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6015d6b0 msgr2=0x7feb600d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 --2- 172.21.15.64:0/583010370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb6015d6b0 0x7feb600d1d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c430 tx=0x7feb5000c8f0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb66aba640 1 -- 172.21.15.64:0/583010370 reap_dead start 2024-03-07T07:20:22.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 shutdown_connections 2024-03-07T07:20:22.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 >> 172.21.15.64:0/583010370 conn(0x7feb6006d970 msgr2=0x7feb600b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 shutdown_connections 2024-03-07T07:20:22.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.226+0000 7feb6851d640 1 -- 172.21.15.64:0/583010370 wait complete. 2024-03-07T07:20:22.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-07T07:20:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.343+0000 7f9c777c9640 1 Processor -- start 2024-03-07T07:20:22.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.343+0000 7f9c777c9640 1 -- start start 2024-03-07T07:20:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c777c9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9c700b5dd0 0x7f9c70144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c777c9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c777c9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c70162c80 con 0x7f9c700b43f0 2024-03-07T07:20:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c777c9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c70141700 con 0x7f9c700b5dd0 2024-03-07T07:20:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c777c9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70071dc0 con 0x7f9c70071410 2024-03-07T07:20:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c75d66640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c75565640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9c700b5dd0 0x7f9c70144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c74d64640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f9c700b43f0 0x7f9c70162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:51268/0 (socket says 172.21.15.64:51268) 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c75565640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9c700b5dd0 0x7f9c70144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:52962/0 (socket says 172.21.15.64:52962) 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c74d64640 1 -- 172.21.15.64:0/716049349 learned_addr learned my addr 172.21.15.64:0/716049349 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.344+0000 7f9c75d66640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:41716/0 (socket says 172.21.15.64:41716) 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.345+0000 7f9c75565640 1 -- 172.21.15.64:0/716049349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 msgr2=0x7f9c70071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.345+0000 7f9c75565640 1 --2- 172.21.15.64:0/716049349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.345+0000 7f9c75565640 1 -- 172.21.15.64:0/716049349 >> v1:172.21.15.120:6789/0 conn(0x7f9c700b43f0 legacy=0x7f9c70162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.345+0000 7f9c75565640 1 -- 172.21.15.64:0/716049349 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c700b4bd0 con 0x7f9c700b5dd0 2024-03-07T07:20:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.345+0000 7f9c75565640 1 --2- 172.21.15.64:0/716049349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b5dd0 0x7f9c70144540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c680097a0 tx=0x7f9c6802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcf20da7faf8c028 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c67fff640 1 -- 172.21.15.64:0/716049349 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c680315a0 con 0x7f9c700b5dd0 2024-03-07T07:20:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c67fff640 1 -- 172.21.15.64:0/716049349 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c68037070 con 0x7f9c700b5dd0 2024-03-07T07:20:22.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c67fff640 1 -- 172.21.15.64:0/716049349 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68031960 con 0x7f9c700b5dd0 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 -- 172.21.15.64:0/716049349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b5dd0 msgr2=0x7f9c70144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 --2- 172.21.15.64:0/716049349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b5dd0 0x7f9c70144540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c680097a0 tx=0x7f9c6802b840 comp rx=0 tx=0).stop 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 -- 172.21.15.64:0/716049349 shutdown_connections 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 --2- 172.21.15.64:0/716049349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 --2- 172.21.15.64:0/716049349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b5dd0 0x7f9c70144540 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 -- 172.21.15.64:0/716049349 >> 172.21.15.64:0/716049349 conn(0x7f9c700d0e70 msgr2=0x7f9c700d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 -- 172.21.15.64:0/716049349 shutdown_connections 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.346+0000 7f9c777c9640 1 -- 172.21.15.64:0/716049349 wait complete. 2024-03-07T07:20:22.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.347+0000 7f9c777c9640 1 Processor -- start 2024-03-07T07:20:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.347+0000 7f9c777c9640 1 -- start start 2024-03-07T07:20:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c777c9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c777c9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b43f0 0x7f9c7013ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c777c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c700b5dd0 0x7f9c7015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c777c9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c700b5190 con 0x7f9c700b5dd0 2024-03-07T07:20:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c777c9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70161e00 con 0x7f9c700b43f0 2024-03-07T07:20:22.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c777c9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70162050 con 0x7f9c70071410 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c75d66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c700b5dd0 0x7f9c7015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c75d66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c700b5dd0 0x7f9c7015b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33270/0 (socket says 172.21.15.64:33270) 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c75d66640 1 -- 172.21.15.64:0/689146393 learned_addr learned my addr 172.21.15.64:0/689146393 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c75565640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.348+0000 7f9c74d64640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b43f0 0x7f9c7013ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c75d66640 1 -- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 msgr2=0x7f9c70158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c75d66640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c75d66640 1 -- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b43f0 msgr2=0x7f9c7013ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c75d66640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b43f0 0x7f9c7013ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c75d66640 1 -- 172.21.15.64:0/689146393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c701f8980 con 0x7f9c700b5dd0 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c75d66640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c700b5dd0 0x7f9c7015b440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009dd0 tx=0x7f9c6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c65ffb640 1 -- 172.21.15.64:0/689146393 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c018020 con 0x7f9c700b5dd0 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c65ffb640 1 -- 172.21.15.64:0/689146393 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c014480 con 0x7f9c700b5dd0 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c65ffb640 1 -- 172.21.15.64:0/689146393 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c014870 con 0x7f9c700b5dd0 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c701f7bc0 con 0x7f9c700b5dd0 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.349+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c701f8180 con 0x7f9c700b5dd0 2024-03-07T07:20:22.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.350+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c70062c90 con 0x7f9c700b5dd0 2024-03-07T07:20:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.350+0000 7f9c65ffb640 1 -- 172.21.15.64:0/689146393 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9c6c005d10 con 0x7f9c700b5dd0 2024-03-07T07:20:22.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.351+0000 7f9c65ffb640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9c5406b500 0x7f9c5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.351+0000 7f9c65ffb640 1 -- 172.21.15.64:0/689146393 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c6c08abd0 con 0x7f9c700b5dd0 2024-03-07T07:20:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.351+0000 7f9c75565640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9c5406b500 0x7f9c5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.352+0000 7f9c75565640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9c5406b500 0x7f9c5406d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c68036000 tx=0x7f9c6803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.355+0000 7f9c65ffb640 1 -- 172.21.15.64:0/689146393 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c6c022070 con 0x7f9c700b5dd0 2024-03-07T07:20:22.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.535+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c7013f600 con 0x7f9c5406b500 2024-03-07T07:20:22.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.536+0000 7f9c65ffb640 1 -- 172.21.15.64:0/689146393 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9c7013f600 con 0x7f9c5406b500 2024-03-07T07:20:22.536 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 1 to deep-scrub 2024-03-07T07:20:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9c5406b500 msgr2=0x7f9c5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9c5406b500 0x7f9c5406d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c68036000 tx=0x7f9c6803d040 comp rx=0 tx=0).stop 2024-03-07T07:20:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c700b5dd0 msgr2=0x7f9c7015b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c700b5dd0 0x7f9c7015b440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009dd0 tx=0x7f9c6c00c490 comp rx=0 tx=0).stop 2024-03-07T07:20:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 shutdown_connections 2024-03-07T07:20:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f9c5406b500 0x7f9c5406d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c700b5dd0 0x7f9c7015b440 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c700b43f0 0x7f9c7013ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 --2- 172.21.15.64:0/689146393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c70071410 0x7f9c70158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 >> 172.21.15.64:0/689146393 conn(0x7f9c700d0e70 msgr2=0x7f9c700d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 shutdown_connections 2024-03-07T07:20:22.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.538+0000 7f9c777c9640 1 -- 172.21.15.64:0/689146393 wait complete. 2024-03-07T07:20:22.549 INFO:tasks.ceph:Scrubbing osd.2 2024-03-07T07:20:22.549 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.655+0000 7f1c579a7640 1 Processor -- start 2024-03-07T07:20:22.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.655+0000 7f1c579a7640 1 -- start start 2024-03-07T07:20:22.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c579a7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c5015bdd0 0x7f1c500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c579a7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c500b56a0 con 0x7f1c5015f350 2024-03-07T07:20:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c579a7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c50071800 con 0x7f1c5006b510 2024-03-07T07:20:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c579a7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5015ff10 con 0x7f1c5015bdd0 2024-03-07T07:20:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c55743640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f1c5015bdd0 0x7f1c500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c54f42640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1c5015f350 0x7f1c5015f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52450/0 (socket says 172.21.15.64:52450) 2024-03-07T07:20:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c54f42640 1 -- 172.21.15.64:0/1209076279 learned_addr learned my addr 172.21.15.64:0/1209076279 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:22.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c55743640 1 -- 172.21.15.64:0/1209076279 >> v1:172.21.15.120:6789/0 conn(0x7f1c5006b510 legacy=0x7f1c500722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c55743640 1 -- 172.21.15.64:0/1209076279 >> v1:172.21.15.64:6789/0 conn(0x7f1c5015f350 legacy=0x7f1c5015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c43fff640 1 -- 172.21.15.64:0/1209076279 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172152112 0 0) 0x7f1c500b56a0 con 0x7f1c5015f350 2024-03-07T07:20:22.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.656+0000 7f1c55743640 1 -- 172.21.15.64:0/1209076279 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c50071ae0 con 0x7f1c5015bdd0 2024-03-07T07:20:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.657+0000 7f1c55743640 1 --2- 172.21.15.64:0/1209076279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5015bdd0 0x7f1c500b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc13d89f25b301a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c43fff640 1 -- 172.21.15.64:0/1209076279 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c034070 con 0x7f1c5015bdd0 2024-03-07T07:20:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c43fff640 1 -- 172.21.15.64:0/1209076279 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c030610 con 0x7f1c5015bdd0 2024-03-07T07:20:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c43fff640 1 -- 172.21.15.64:0/1209076279 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0309e0 con 0x7f1c5015bdd0 2024-03-07T07:20:22.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c579a7640 1 -- 172.21.15.64:0/1209076279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5015bdd0 msgr2=0x7f1c500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c579a7640 1 --2- 172.21.15.64:0/1209076279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5015bdd0 0x7f1c500b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02b960 comp rx=0 tx=0).stop 2024-03-07T07:20:22.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c579a7640 1 -- 172.21.15.64:0/1209076279 shutdown_connections 2024-03-07T07:20:22.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c579a7640 1 --2- 172.21.15.64:0/1209076279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5015bdd0 0x7f1c500b4d90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c579a7640 1 -- 172.21.15.64:0/1209076279 >> 172.21.15.64:0/1209076279 conn(0x7f1c50066e50 msgr2=0x7f1c50067260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c579a7640 1 -- 172.21.15.64:0/1209076279 shutdown_connections 2024-03-07T07:20:22.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.658+0000 7f1c579a7640 1 -- 172.21.15.64:0/1209076279 wait complete. 2024-03-07T07:20:22.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.660+0000 7f1c579a7640 1 Processor -- start 2024-03-07T07:20:22.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.660+0000 7f1c579a7640 1 -- start start 2024-03-07T07:20:22.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c579a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5006b510 0x7f1c5014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c579a7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c5015bdd0 0x7f1c501504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c579a7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c5015f350 0x7f1c501562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c579a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500b5b00 con 0x7f1c5006b510 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c579a7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500b5980 con 0x7f1c5015bdd0 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c579a7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500729f0 con 0x7f1c5015f350 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c55743640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5006b510 0x7f1c5014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c55743640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5006b510 0x7f1c5014fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33288/0 (socket says 172.21.15.64:33288) 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.661+0000 7f1c55743640 1 -- 172.21.15.64:0/2186325309 learned_addr learned my addr 172.21.15.64:0/2186325309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c54f42640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c5015bdd0 0x7f1c501504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55743640 1 -- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c5015f350 msgr2=0x7f1c501562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55f44640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c5015f350 0x7f1c501562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55743640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c5015f350 0x7f1c501562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55743640 1 -- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c5015bdd0 msgr2=0x7f1c501504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55743640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c5015bdd0 0x7f1c501504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55743640 1 -- 172.21.15.64:0/2186325309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c50150fa0 con 0x7f1c5006b510 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c54f42640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c5015bdd0 0x7f1c501504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55f44640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1c5015f350 0x7f1c501562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c55743640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5006b510 0x7f1c5014fe40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0098c0 tx=0x7f1c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c034040 con 0x7f1c5006b510 2024-03-07T07:20:22.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c039a10 con 0x7f1c5006b510 2024-03-07T07:20:22.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c50151280 con 0x7f1c5006b510 2024-03-07T07:20:22.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c039de0 con 0x7f1c5006b510 2024-03-07T07:20:22.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.662+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c50151840 con 0x7f1c5006b510 2024-03-07T07:20:22.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.663+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c50062c90 con 0x7f1c5006b510 2024-03-07T07:20:22.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.664+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1c4c037070 con 0x7f1c5006b510 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.664+0000 7f1c41ffb640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1c3406b500 0x7f1c3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.664+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c4c0acd10 con 0x7f1c5006b510 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.664+0000 7f1c41ffb640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f1c340736a0 0x7f1c34075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.664+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 --> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c4c03ae50 con 0x7f1c340736a0 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.664+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c4c0ad1a0 con 0x7f1c5006b510 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.664+0000 7f1c54f42640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1c3406b500 0x7f1c3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.665+0000 7f1c55f44640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f1c340736a0 0x7f1c34075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.665+0000 7f1c55f44640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f1c340736a0 0x7f1c34075ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.665+0000 7f1c54f42640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1c3406b500 0x7f1c3406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1c50156f40 tx=0x7f1c44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.665+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== osd.2 v2:172.21.15.64:6816/3037744413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1c4c03ae50 con 0x7f1c340736a0 2024-03-07T07:20:22.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 --> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1c50074690 con 0x7f1c340736a0 2024-03-07T07:20:22.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c41ffb640 1 -- 172.21.15.64:0/2186325309 <== osd.2 v2:172.21.15.64:6816/3037744413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1c50074690 con 0x7f1c340736a0 2024-03-07T07:20:22.679 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:22.679 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:22.680 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:22.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f1c340736a0 msgr2=0x7f1c34075ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:6816/3037744413,v1:172.21.15.64:6817/3037744413] conn(0x7f1c340736a0 0x7f1c34075ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1c3406b500 msgr2=0x7f1c3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f1c3406b500 0x7f1c3406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1c50156f40 tx=0x7f1c44009040 comp rx=0 tx=0).stop 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5006b510 msgr2=0x7f1c5014fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 --2- 172.21.15.64:0/2186325309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c5006b510 0x7f1c5014fe40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0098c0 tx=0x7f1c4c0046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c55f44640 1 -- 172.21.15.64:0/2186325309 reap_dead start 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 shutdown_connections 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.679+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 >> 172.21.15.64:0/2186325309 conn(0x7f1c50066e50 msgr2=0x7f1c5015aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.680+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 shutdown_connections 2024-03-07T07:20:22.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.680+0000 7f1c579a7640 1 -- 172.21.15.64:0/2186325309 wait complete. 2024-03-07T07:20:22.690 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-07T07:20:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.802+0000 7f95f46dd640 1 Processor -- start 2024-03-07T07:20:22.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.802+0000 7f95f46dd640 1 -- start start 2024-03-07T07:20:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.803+0000 7f95f46dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 0x7f95ec162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.803+0000 7f95f46dd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec0b4bd0 con 0x7f95ec0b5dd0 2024-03-07T07:20:22.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.803+0000 7f95f46dd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec0b4eb0 con 0x7f95ec162ab0 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.803+0000 7f95f46dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec141700 con 0x7f95ec144540 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.803+0000 7f95f2479640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f95ec0b5dd0 0x7f95ec0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52460/0 (socket says 172.21.15.64:52460) 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.803+0000 7f95f2479640 1 -- 172.21.15.64:0/410206153 learned_addr learned my addr 172.21.15.64:0/410206153 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.803+0000 7f95f1c78640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 0x7f95ec162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558058310 0 0) 0x7f95ec0b4bd0 con 0x7f95ec0b5dd0 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d8003520 con 0x7f95ec0b5dd0 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326987804 0 0) 0x7f95d8003520 con 0x7f95ec0b5dd0 2024-03-07T07:20:22.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 msgr2=0x7f95ec162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 0x7f95ec162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 >> v1:172.21.15.120:6789/0 conn(0x7f95ec162ab0 legacy=0x7f95ec071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0b5190 con 0x7f95ec0b5dd0 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f95e8001670 con 0x7f95ec0b5dd0 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 >> v1:172.21.15.64:6789/0 conn(0x7f95ec0b5dd0 legacy=0x7f95ec0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d8004390 0x7f95d8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 0x7f95d8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec141700 con 0x7f95d80039e0 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d800ba70 con 0x7f95d8004390 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d800bc70 con 0x7f95ec144540 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f2479640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f1c78640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d8004390 0x7f95d8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f2c7a640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 0x7f95d8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f2479640 1 -- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 msgr2=0x7f95d8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:22.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f2479640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec144540 0x7f95d8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f2479640 1 -- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d8004390 msgr2=0x7f95d8004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f2479640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95d8004390 0x7f95d8004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.804+0000 7f95f2479640 1 -- 172.21.15.64:0/410206153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d8003520 con 0x7f95d80039e0 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.805+0000 7f95f2479640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95d8003dc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007fd0 tx=0x7f95d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a53a9f50c5a71b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.805+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8007ca0 con 0x7f95d80039e0 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.805+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e8030070 con 0x7f95d80039e0 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.805+0000 7f95f1477640 1 -- 172.21.15.64:0/410206153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8005d10 con 0x7f95d80039e0 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.806+0000 7f95f46dd640 1 -- 172.21.15.64:0/410206153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 msgr2=0x7f95d8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.806+0000 7f95f46dd640 1 --2- 172.21.15.64:0/410206153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95d8003dc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007fd0 tx=0x7f95d8005830 comp rx=0 tx=0).stop 2024-03-07T07:20:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.806+0000 7f95f2c7a640 1 -- 172.21.15.64:0/410206153 reap_dead start 2024-03-07T07:20:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.806+0000 7f95f46dd640 1 -- 172.21.15.64:0/410206153 shutdown_connections 2024-03-07T07:20:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.806+0000 7f95f46dd640 1 -- 172.21.15.64:0/410206153 >> 172.21.15.64:0/410206153 conn(0x7f95ec0d0e70 msgr2=0x7f95ec0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.806+0000 7f95f46dd640 1 -- 172.21.15.64:0/410206153 shutdown_connections 2024-03-07T07:20:22.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.806+0000 7f95f46dd640 1 -- 172.21.15.64:0/410206153 wait complete. 2024-03-07T07:20:22.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.807+0000 7f95f46dd640 1 Processor -- start 2024-03-07T07:20:22.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.808+0000 7f95f46dd640 1 -- start start 2024-03-07T07:20:22.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f46dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95ec15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f46dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95ec15bad0 0x7f95ec160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f46dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec1585b0 0x7f95ec161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f46dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec0b61b0 con 0x7f95d80039e0 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f46dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec161e00 con 0x7f95ec15bad0 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f46dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec071b20 con 0x7f95ec1585b0 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95ec15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95ec15b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33306/0 (socket says 172.21.15.64:33306) 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 -- 172.21.15.64:0/246343963 learned_addr learned my addr 172.21.15.64:0/246343963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f1c78640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95ec15bad0 0x7f95ec160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2c7a640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec1585b0 0x7f95ec161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 -- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec1585b0 msgr2=0x7f95ec161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec1585b0 0x7f95ec161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 -- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95ec15bad0 msgr2=0x7f95ec160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95ec15bad0 0x7f95ec160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 -- 172.21.15.64:0/246343963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec15a150 con 0x7f95d80039e0 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95f2479640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95ec15b590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c220 tx=0x7f95ec0b5080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.809+0000 7f95deffd640 1 -- 172.21.15.64:0/246343963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8045070 con 0x7f95d80039e0 2024-03-07T07:20:22.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.810+0000 7f95deffd640 1 -- 172.21.15.64:0/246343963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e8007c80 con 0x7f95d80039e0 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.810+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec15a430 con 0x7f95d80039e0 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.810+0000 7f95deffd640 1 -- 172.21.15.64:0/246343963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8001bd0 con 0x7f95d80039e0 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.810+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec15a9f0 con 0x7f95d80039e0 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.810+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec062c90 con 0x7f95d80039e0 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.811+0000 7f95deffd640 1 -- 172.21.15.64:0/246343963 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f95e8004c50 con 0x7f95d80039e0 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.811+0000 7f95deffd640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f95d006b500 0x7f95d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.811+0000 7f95deffd640 1 -- 172.21.15.64:0/246343963 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95e80aaf40 con 0x7f95d80039e0 2024-03-07T07:20:22.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.812+0000 7f95f1c78640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f95d006b500 0x7f95d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.812+0000 7f95f1c78640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f95d006b500 0x7f95d006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f95ec15df20 tx=0x7f95e0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:22.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.816+0000 7f95deffd640 1 -- 172.21.15.64:0/246343963 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95e80798e0 con 0x7f95d80039e0 2024-03-07T07:20:22.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.997+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f95ec0b8a10 con 0x7f95d006b500 2024-03-07T07:20:22.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:22.998+0000 7f95deffd640 1 -- 172.21.15.64:0/246343963 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f95ec0b8a10 con 0x7f95d006b500 2024-03-07T07:20:22.998 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 2 to deep-scrub 2024-03-07T07:20:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f95d006b500 msgr2=0x7f95d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f95d006b500 0x7f95d006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f95ec15df20 tx=0x7f95e0009190 comp rx=0 tx=0).stop 2024-03-07T07:20:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 msgr2=0x7f95ec15b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95ec15b590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c220 tx=0x7f95ec0b5080 comp rx=0 tx=0).stop 2024-03-07T07:20:23.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 shutdown_connections 2024-03-07T07:20:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f95d006b500 0x7f95d006d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f95ec1585b0 0x7f95ec161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f95ec15bad0 0x7f95ec160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 --2- 172.21.15.64:0/246343963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95d80039e0 0x7f95ec15b590 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 >> 172.21.15.64:0/246343963 conn(0x7f95ec0d0e70 msgr2=0x7f95ec1401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 shutdown_connections 2024-03-07T07:20:23.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.000+0000 7f95f46dd640 1 -- 172.21.15.64:0/246343963 wait complete. 2024-03-07T07:20:23.013 INFO:tasks.ceph:Scrubbing osd.3 2024-03-07T07:20:23.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:23.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.157+0000 7f3e10434640 1 Processor -- start 2024-03-07T07:20:23.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.158+0000 7f3e10434640 1 -- start start 2024-03-07T07:20:23.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e10434640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e10434640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3e080b4970 0x7f3e080b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e10434640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e080b2440 con 0x7f3e080cf9e0 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e10434640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e080b2720 con 0x7f3e080b4970 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e10434640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0815add0 con 0x7f3e08157ec0 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0e9d1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3e080b4970 0x7f3e080b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0d9cf640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3e080cf9e0 0x7f3e080b1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52476/0 (socket says 172.21.15.64:52476) 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0d9cf640 1 -- 172.21.15.64:0/2620307860 learned_addr learned my addr 172.21.15.64:0/2620307860 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0e1d0640 1 --2- 172.21.15.64:0/2620307860 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0e9d1640 1 -- 172.21.15.64:0/2620307860 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 msgr2=0x7f3e0815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0e9d1640 1 --2- 172.21.15.64:0/2620307860 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0815b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0e9d1640 1 -- 172.21.15.64:0/2620307860 >> v1:172.21.15.64:6789/0 conn(0x7f3e080cf9e0 legacy=0x7f3e080b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0d1ce640 1 -- 172.21.15.64:0/2620307860 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455781985 0 0) 0x7f3e080b2440 con 0x7f3e080cf9e0 2024-03-07T07:20:23.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.159+0000 7f3e0e9d1640 1 -- 172.21.15.64:0/2620307860 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0815b0b0 con 0x7f3e080b4970 2024-03-07T07:20:23.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.160+0000 7f3e0e9d1640 1 --2- 172.21.15.64:0/2620307860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080b4970 0x7f3e080b4e00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000aea0 tx=0x7f3e0002cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47da8e981c26503d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.160+0000 7f3e0d1ce640 1 -- 172.21.15.64:0/2620307860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0002db40 con 0x7f3e080b4970 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.160+0000 7f3e0d1ce640 1 -- 172.21.15.64:0/2620307860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0002dce0 con 0x7f3e080b4970 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.160+0000 7f3e0d1ce640 1 -- 172.21.15.64:0/2620307860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00008660 con 0x7f3e080b4970 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.160+0000 7f3e10434640 1 -- 172.21.15.64:0/2620307860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080b4970 msgr2=0x7f3e080b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.160+0000 7f3e10434640 1 --2- 172.21.15.64:0/2620307860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080b4970 0x7f3e080b4e00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000aea0 tx=0x7f3e0002cfa0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.161+0000 7f3e10434640 1 -- 172.21.15.64:0/2620307860 shutdown_connections 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.161+0000 7f3e10434640 1 --2- 172.21.15.64:0/2620307860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080b4970 0x7f3e080b4e00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.161+0000 7f3e10434640 1 --2- 172.21.15.64:0/2620307860 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.161+0000 7f3e10434640 1 -- 172.21.15.64:0/2620307860 >> 172.21.15.64:0/2620307860 conn(0x7f3e08066eb0 msgr2=0x7f3e08067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.161+0000 7f3e10434640 1 -- 172.21.15.64:0/2620307860 shutdown_connections 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.161+0000 7f3e10434640 1 -- 172.21.15.64:0/2620307860 wait complete. 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 Processor -- start 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 -- start start 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e080b4970 0x7f3e081f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080cf9e0 0x7f3e0814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08158420 con 0x7f3e080cf9e0 2024-03-07T07:20:23.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081582a0 con 0x7f3e080b4970 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.162+0000 7f3e10434640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08070820 con 0x7f3e08157ec0 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080cf9e0 0x7f3e0814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080cf9e0 0x7f3e0814cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33324/0 (socket says 172.21.15.64:33324) 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 -- 172.21.15.64:0/2779418032 learned_addr learned my addr 172.21.15.64:0/2779418032 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0e9d1640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0e1d0640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e080b4970 0x7f3e081f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 -- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 msgr2=0x7f3e0814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 -- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e080b4970 msgr2=0x7f3e081f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e080b4970 0x7f3e081f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 -- 172.21.15.64:0/2779418032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0814d1e0 con 0x7f3e080cf9e0 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0e9d1640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3e08157ec0 0x7f3e0814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e0d9cf640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080cf9e0 0x7f3e0814cca0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c4d0 tx=0x7f3df800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8018070 con 0x7f3e080cf9e0 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df8007500 con 0x7f3e080cf9e0 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0813d8b0 con 0x7f3e080cf9e0 2024-03-07T07:20:23.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8004a20 con 0x7f3e080cf9e0 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.163+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0813de70 con 0x7f3e080cf9e0 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.164+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e08062c90 con 0x7f3e080cf9e0 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.165+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3df801b070 con 0x7f3e080cf9e0 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.165+0000 7f3dfeffd640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3dec06b4d0 0x7f3dec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.165+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3df80899e0 con 0x7f3e080cf9e0 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.165+0000 7f3dfeffd640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7f3dec0736d0 0x7f3dec075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.165+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 --> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3dec0761a0 con 0x7f3dec0736d0 2024-03-07T07:20:23.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.165+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df8089e70 con 0x7f3e080cf9e0 2024-03-07T07:20:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.166+0000 7f3e0e9d1640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7f3dec0736d0 0x7f3dec075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.166+0000 7f3e0e1d0640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3dec06b4d0 0x7f3dec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.166+0000 7f3e0e9d1640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7f3dec0736d0 0x7f3dec075af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.166+0000 7f3e0e1d0640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3dec06b4d0 0x7f3dec06d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e040079a0 tx=0x7f3e04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.167+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== osd.3 v2:172.21.15.64:6824/2270479097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3dec0761a0 con 0x7f3dec0736d0 2024-03-07T07:20:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.180+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 --> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3e0806e8e0 con 0x7f3dec0736d0 2024-03-07T07:20:23.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.180+0000 7f3dfeffd640 1 -- 172.21.15.64:0/2779418032 <== osd.3 v2:172.21.15.64:6824/2270479097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3e0806e8e0 con 0x7f3dec0736d0 2024-03-07T07:20:23.181 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:23.181 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:23.181 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:23.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7f3dec0736d0 msgr2=0x7f3dec075af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:6824/2270479097,v1:172.21.15.64:6825/2270479097] conn(0x7f3dec0736d0 0x7f3dec075af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3dec06b4d0 msgr2=0x7f3dec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3dec06b4d0 0x7f3dec06d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e040079a0 tx=0x7f3e04008040 comp rx=0 tx=0).stop 2024-03-07T07:20:23.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080cf9e0 msgr2=0x7f3e0814cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 --2- 172.21.15.64:0/2779418032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e080cf9e0 0x7f3e0814cca0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c4d0 tx=0x7f3df800c990 comp rx=0 tx=0).stop 2024-03-07T07:20:23.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e0e9d1640 1 -- 172.21.15.64:0/2779418032 reap_dead start 2024-03-07T07:20:23.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 shutdown_connections 2024-03-07T07:20:23.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 >> 172.21.15.64:0/2779418032 conn(0x7f3e08066eb0 msgr2=0x7f3e0806eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 shutdown_connections 2024-03-07T07:20:23.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.181+0000 7f3e10434640 1 -- 172.21.15.64:0/2779418032 wait complete. 2024-03-07T07:20:23.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-07T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.294+0000 7f032211d640 1 Processor -- start 2024-03-07T07:20:23.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.294+0000 7f032211d640 1 -- start start 2024-03-07T07:20:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f032211d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c13dd70 0x7f031c15f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f032211d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f031c15ea40 0x7f031c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f032211d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c13e150 con 0x7f031c0b3010 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f032211d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031c06f000 con 0x7f031c15ea40 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f032211d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c06e3f0 con 0x7f031c13dd70 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f031b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c13dd70 0x7f031c15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f031affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f031c15ea40 0x7f031c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f031bfff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f031c0b3010 0x7f031c140510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:51296/0 (socket says 172.21.15.64:51296) 2024-03-07T07:20:23.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.295+0000 7f031bfff640 1 -- 172.21.15.64:0/229561506 learned_addr learned my addr 172.21.15.64:0/229561506 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.296+0000 7f031affd640 1 -- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c13dd70 msgr2=0x7f031c15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.296+0000 7f031affd640 1 --2- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c13dd70 0x7f031c15f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.296+0000 7f031affd640 1 -- 172.21.15.64:0/229561506 >> v1:172.21.15.120:6789/0 conn(0x7f031c0b3010 legacy=0x7f031c140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.296+0000 7f031affd640 1 -- 172.21.15.64:0/229561506 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c06e6d0 con 0x7f031c15ea40 2024-03-07T07:20:23.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.296+0000 7f031b7fe640 1 --2- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c13dd70 0x7f031c15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:23.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.296+0000 7f031affd640 1 --2- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 0x7f031c0b2ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f03100097a0 tx=0x7f031002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa7e4201044f505 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f031a7fc640 1 -- 172.21.15.64:0/229561506 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310035020 con 0x7f031c15ea40 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f031a7fc640 1 -- 172.21.15.64:0/229561506 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0310031830 con 0x7f031c15ea40 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f031a7fc640 1 -- 172.21.15.64:0/229561506 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310031c00 con 0x7f031c15ea40 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 -- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 msgr2=0x7f031c0b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 --2- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 0x7f031c0b2ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f03100097a0 tx=0x7f031002b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 -- 172.21.15.64:0/229561506 shutdown_connections 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 --2- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 0x7f031c0b2ad0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 --2- 172.21.15.64:0/229561506 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c13dd70 0x7f031c15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 -- 172.21.15.64:0/229561506 >> 172.21.15.64:0/229561506 conn(0x7f031c06b040 msgr2=0x7f031c06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 -- 172.21.15.64:0/229561506 shutdown_connections 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.297+0000 7f032211d640 1 -- 172.21.15.64:0/229561506 wait complete. 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.298+0000 7f032211d640 1 Processor -- start 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.298+0000 7f032211d640 1 -- start start 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f032211d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c0b3010 0x7f031c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f032211d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f031c13dd70 0x7f031c156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f032211d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 0x7f031c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f032211d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c15f7b0 con 0x7f031c13dd70 2024-03-07T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f032211d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c06ec90 con 0x7f031c15ea40 2024-03-07T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f032211d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0d07f0 con 0x7f031c0b3010 2024-03-07T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f031affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f031c13dd70 0x7f031c156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f031affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f031c13dd70 0x7f031c156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33328/0 (socket says 172.21.15.64:33328) 2024-03-07T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f031affd640 1 -- 172.21.15.64:0/4099666094 learned_addr learned my addr 172.21.15.64:0/4099666094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f031b7fe640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c0b3010 0x7f031c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.299+0000 7f031bfff640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 0x7f031c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f031affd640 1 -- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c0b3010 msgr2=0x7f031c0706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f031affd640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c0b3010 0x7f031c0706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f031affd640 1 -- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 msgr2=0x7f031c14db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f031affd640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 0x7f031c14db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f031affd640 1 -- 172.21.15.64:0/4099666094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c1f4110 con 0x7f031c13dd70 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f031affd640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f031c13dd70 0x7f031c156a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0310034040 tx=0x7f03100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f0303fff640 1 -- 172.21.15.64:0/4099666094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310040070 con 0x7f031c13dd70 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c1f43f0 con 0x7f031c13dd70 2024-03-07T07:20:23.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f0303fff640 1 -- 172.21.15.64:0/4099666094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031003a910 con 0x7f031c13dd70 2024-03-07T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f0303fff640 1 -- 172.21.15.64:0/4099666094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310035040 con 0x7f031c13dd70 2024-03-07T07:20:23.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.300+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c1f49b0 con 0x7f031c13dd70 2024-03-07T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.301+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031c062c90 con 0x7f031c13dd70 2024-03-07T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.301+0000 7f0303fff640 1 -- 172.21.15.64:0/4099666094 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0310038020 con 0x7f031c13dd70 2024-03-07T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.302+0000 7f0303fff640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f030806b4d0 0x7f030806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.302+0000 7f0303fff640 1 -- 172.21.15.64:0/4099666094 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03100abb40 con 0x7f031c13dd70 2024-03-07T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.302+0000 7f031b7fe640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f030806b4d0 0x7f030806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.303+0000 7f031b7fe640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f030806b4d0 0x7f030806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f030c009560 tx=0x7f030c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.306+0000 7f0303fff640 1 -- 172.21.15.64:0/4099666094 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0310031800 con 0x7f031c13dd70 2024-03-07T07:20:23.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.488+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f031c0b3c30 con 0x7f030806b4d0 2024-03-07T07:20:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.489+0000 7f0303fff640 1 -- 172.21.15.64:0/4099666094 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f031c0b3c30 con 0x7f030806b4d0 2024-03-07T07:20:23.489 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 3 to deep-scrub 2024-03-07T07:20:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f030806b4d0 msgr2=0x7f030806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f030806b4d0 0x7f030806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f030c009560 tx=0x7f030c009190 comp rx=0 tx=0).stop 2024-03-07T07:20:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f031c13dd70 msgr2=0x7f031c156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f031c13dd70 0x7f031c156a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0310034040 tx=0x7f03100046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 shutdown_connections 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f030806b4d0 0x7f030806d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f031c13dd70 0x7f031c156a10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f031c15ea40 0x7f031c14db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 --2- 172.21.15.64:0/4099666094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f031c0b3010 0x7f031c0706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 >> 172.21.15.64:0/4099666094 conn(0x7f031c06b040 msgr2=0x7f031c15db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 shutdown_connections 2024-03-07T07:20:23.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.491+0000 7f032211d640 1 -- 172.21.15.64:0/4099666094 wait complete. 2024-03-07T07:20:23.505 INFO:tasks.ceph:Scrubbing osd.4 2024-03-07T07:20:23.506 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:23.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.614+0000 7f5a0df78640 1 Processor -- start 2024-03-07T07:20:23.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.614+0000 7f5a0df78640 1 -- start start 2024-03-07T07:20:23.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.615+0000 7f5a0df78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 0x7f5a08161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.615+0000 7f5a0df78640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5a08162460 0x7f5a080d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.615+0000 7f5a0df78640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a08162880 con 0x7f5a0815e210 2024-03-07T07:20:23.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.615+0000 7f5a0df78640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080d2640 con 0x7f5a08162460 2024-03-07T07:20:23.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.615+0000 7f5a0df78640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08071880 con 0x7f5a0815e5f0 2024-03-07T07:20:23.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.615+0000 7f5a077fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5a0815e210 0x7f5a080b9a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52492/0 (socket says 172.21.15.64:52492) 2024-03-07T07:20:23.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.615+0000 7f5a077fe640 1 -- 172.21.15.64:0/1524997132 learned_addr learned my addr 172.21.15.64:0/1524997132 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:23.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 0x7f5a08161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a07fff640 1 --2- 172.21.15.64:0/1524997132 >> v2:172.21.15.120:3300/0 conn(0x7f5a08162460 0x7f5a080d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176891087 0 0) 0x7f5a08162880 con 0x7f5a0815e210 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f0003520 con 0x7f5a0815e210 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 347529942 0 0) 0x7f59f0003520 con 0x7f5a0815e210 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 msgr2=0x7f5a08161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 0x7f5a08161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 >> v2:172.21.15.120:3300/0 conn(0x7f5a08162460 msgr2=0x7f5a080d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 --2- 172.21.15.64:0/1524997132 >> v2:172.21.15.120:3300/0 conn(0x7f5a08162460 0x7f5a080d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08071b60 con 0x7f5a0815e210 2024-03-07T07:20:23.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f59f8001670 con 0x7f5a0815e210 2024-03-07T07:20:23.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 >> v1:172.21.15.64:6789/0 conn(0x7f5a0815e210 legacy=0x7f5a080b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 0x7f59f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59f0003f30 0x7f59f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f0004d40 0x7f59f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080d2640 con 0x7f59f0003f30 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08071880 con 0x7f59f0004d40 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f000b9c0 con 0x7f5a0815e5f0 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59f0003f30 0x7f59f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a077fe640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 0x7f59f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.616+0000 7f5a07fff640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f0004d40 0x7f59f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a06ffd640 1 -- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 msgr2=0x7f59f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a0815e5f0 0x7f59f00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a06ffd640 1 -- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f0004d40 msgr2=0x7f59f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59f0004d40 0x7f59f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a06ffd640 1 -- 172.21.15.64:0/1524997132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0003520 con 0x7f59f0003f30 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59f0003f30 0x7f59f0004800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009a90 tx=0x7f59f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f49e3119a8fa404 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc039070 con 0x7f59f0003f30 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc02c6b0 con 0x7f59f0003f30 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.617+0000 7f5a067fc640 1 -- 172.21.15.64:0/1524997132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc009690 con 0x7f59f0003f30 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.618+0000 7f5a0df78640 1 -- 172.21.15.64:0/1524997132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59f0003f30 msgr2=0x7f59f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.618+0000 7f5a0df78640 1 --2- 172.21.15.64:0/1524997132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59f0003f30 0x7f59f0004800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009a90 tx=0x7f59f0005830 comp rx=0 tx=0).stop 2024-03-07T07:20:23.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.618+0000 7f5a07fff640 1 -- 172.21.15.64:0/1524997132 reap_dead start 2024-03-07T07:20:23.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.618+0000 7f5a0df78640 1 -- 172.21.15.64:0/1524997132 shutdown_connections 2024-03-07T07:20:23.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.618+0000 7f5a0df78640 1 -- 172.21.15.64:0/1524997132 >> 172.21.15.64:0/1524997132 conn(0x7f5a080d40b0 msgr2=0x7f5a080d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.618+0000 7f5a0df78640 1 -- 172.21.15.64:0/1524997132 shutdown_connections 2024-03-07T07:20:23.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.618+0000 7f5a0df78640 1 -- 172.21.15.64:0/1524997132 wait complete. 2024-03-07T07:20:23.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.620+0000 7f5a0df78640 1 Processor -- start 2024-03-07T07:20:23.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.620+0000 7f5a0df78640 1 -- start start 2024-03-07T07:20:23.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a0df78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59f0003f30 0x7f5a0815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a0df78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0814ec10 0x7f5a080cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a0df78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a080cc780 0x7f5a081f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a0df78640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08071410 con 0x7f5a0814ec10 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a0df78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08071e40 con 0x7f5a080cc780 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a0df78640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08071590 con 0x7f59f0003f30 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0814ec10 0x7f5a080cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0814ec10 0x7f5a080cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33356/0 (socket says 172.21.15.64:33356) 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.621+0000 7f5a06ffd640 1 -- 172.21.15.64:0/9761714 learned_addr learned my addr 172.21.15.64:0/9761714 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a06ffd640 1 -- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59f0003f30 msgr2=0x7f5a0815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59f0003f30 0x7f5a0815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a07fff640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a080cc780 0x7f5a081f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a06ffd640 1 -- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a080cc780 msgr2=0x7f5a081f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a080cc780 0x7f5a081f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a06ffd640 1 -- 172.21.15.64:0/9761714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0815c760 con 0x7f5a0814ec10 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a06ffd640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0814ec10 0x7f5a080cc240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f59fc000c00 tx=0x7f59fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc042020 con 0x7f5a0814ec10 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc03a040 con 0x7f5a0814ec10 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0815c9e0 con 0x7f5a0814ec10 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc039070 con 0x7f5a0814ec10 2024-03-07T07:20:23.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.622+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0815b310 con 0x7f5a0814ec10 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.623+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a08074810 con 0x7f5a0814ec10 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.623+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f59fc045020 con 0x7f5a0814ec10 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.624+0000 7f59effff640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f59f406b4d0 0x7f59f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.624+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59fc0ab640 con 0x7f5a0814ec10 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.624+0000 7f59effff640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f59f40736d0 0x7f59f4075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.624+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 --> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59f40761a0 con 0x7f59f40736d0 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.624+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59fc0b2050 con 0x7f5a0814ec10 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.624+0000 7f5a077fe640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f59f406b4d0 0x7f59f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.624+0000 7f5a07fff640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f59f40736d0 0x7f59f4075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.625+0000 7f5a077fe640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f59f406b4d0 0x7f59f406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f59f8001be0 tx=0x7f59f8001b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.625+0000 7f5a07fff640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f59f40736d0 0x7f59f4075af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.625+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== osd.4 v2:172.21.15.120:6808/505090945 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f59f40761a0 con 0x7f59f40736d0 2024-03-07T07:20:23.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 --> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5a08062c90 con 0x7f59f40736d0 2024-03-07T07:20:23.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f59effff640 1 -- 172.21.15.64:0/9761714 <== osd.4 v2:172.21.15.120:6808/505090945 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5a08062c90 con 0x7f59f40736d0 2024-03-07T07:20:23.639 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:23.639 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:23.639 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:23.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f59f40736d0 msgr2=0x7f59f4075af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6808/505090945,v1:172.21.15.120:6809/505090945] conn(0x7f59f40736d0 0x7f59f4075af0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f59f406b4d0 msgr2=0x7f59f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f59f406b4d0 0x7f59f406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f59f8001be0 tx=0x7f59f8001b70 comp rx=0 tx=0).stop 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0814ec10 msgr2=0x7f5a080cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 --2- 172.21.15.64:0/9761714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0814ec10 0x7f5a080cc240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f59fc000c00 tx=0x7f59fc004730 comp rx=0 tx=0).stop 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a07fff640 1 -- 172.21.15.64:0/9761714 reap_dead start 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 shutdown_connections 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 >> 172.21.15.64:0/9761714 conn(0x7f5a080d40b0 msgr2=0x7f5a0815ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 shutdown_connections 2024-03-07T07:20:23.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.639+0000 7f5a0df78640 1 -- 172.21.15.64:0/9761714 wait complete. 2024-03-07T07:20:23.650 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-07T07:20:23.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.752+0000 7fad9211d640 1 Processor -- start 2024-03-07T07:20:23.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.752+0000 7fad9211d640 1 -- start start 2024-03-07T07:20:23.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad9211d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fad8c13dd70 0x7fad8c15f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad9211d640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad8c15ea40 0x7fad8c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad9211d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c0b3010 0x7fad8c140510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad9211d640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c0b3480 con 0x7fad8c15ea40 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad9211d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06f000 con 0x7fad8c13dd70 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad9211d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c13e150 con 0x7fad8c0b3010 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad8c15ea40 0x7fad8c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fad8c15ea40 0x7fad8c0b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33366/0 (socket says 172.21.15.64:33366) 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 -- 172.21.15.64:0/2874085238 learned_addr learned my addr 172.21.15.64:0/2874085238 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8b7fe640 1 --2- 172.21.15.64:0/2874085238 >> v2:172.21.15.120:3300/0 conn(0x7fad8c13dd70 0x7fad8c15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8bfff640 1 --2- 172.21.15.64:0/2874085238 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c0b3010 0x7fad8c140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 -- 172.21.15.64:0/2874085238 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c0b3010 msgr2=0x7fad8c140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 --2- 172.21.15.64:0/2874085238 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c0b3010 0x7fad8c140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 -- 172.21.15.64:0/2874085238 >> v2:172.21.15.120:3300/0 conn(0x7fad8c13dd70 msgr2=0x7fad8c15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 --2- 172.21.15.64:0/2874085238 >> v2:172.21.15.120:3300/0 conn(0x7fad8c13dd70 0x7fad8c15f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8affd640 1 -- 172.21.15.64:0/2874085238 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c06e3f0 con 0x7fad8c15ea40 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.753+0000 7fad8b7fe640 1 --2- 172.21.15.64:0/2874085238 >> v2:172.21.15.120:3300/0 conn(0x7fad8c13dd70 0x7fad8c15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.754+0000 7fad8affd640 1 --2- 172.21.15.64:0/2874085238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c15ea40 0x7fad8c0b2ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0b6dc0 tx=0x7fad8002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a81defb5ea5e683 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.754+0000 7fad8a7fc640 1 -- 172.21.15.64:0/2874085238 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8003b070 con 0x7fad8c15ea40 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.754+0000 7fad8a7fc640 1 -- 172.21.15.64:0/2874085238 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80036440 con 0x7fad8c15ea40 2024-03-07T07:20:23.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.754+0000 7fad8a7fc640 1 -- 172.21.15.64:0/2874085238 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80036810 con 0x7fad8c15ea40 2024-03-07T07:20:23.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 -- 172.21.15.64:0/2874085238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c15ea40 msgr2=0x7fad8c0b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 --2- 172.21.15.64:0/2874085238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c15ea40 0x7fad8c0b2ad0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0b6dc0 tx=0x7fad8002e0e0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 -- 172.21.15.64:0/2874085238 shutdown_connections 2024-03-07T07:20:23.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 --2- 172.21.15.64:0/2874085238 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c0b3010 0x7fad8c140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 --2- 172.21.15.64:0/2874085238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c15ea40 0x7fad8c0b2ad0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 --2- 172.21.15.64:0/2874085238 >> v2:172.21.15.120:3300/0 conn(0x7fad8c13dd70 0x7fad8c15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 -- 172.21.15.64:0/2874085238 >> 172.21.15.64:0/2874085238 conn(0x7fad8c06b040 msgr2=0x7fad8c06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 -- 172.21.15.64:0/2874085238 shutdown_connections 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.755+0000 7fad9211d640 1 -- 172.21.15.64:0/2874085238 wait complete. 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.756+0000 7fad9211d640 1 Processor -- start 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.756+0000 7fad9211d640 1 -- start start 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad9211d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c0b3010 0x7fad8c0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad9211d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c13dd70 0x7fad8c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad9211d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c15ea40 0x7fad8c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad9211d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c06ec90 con 0x7fad8c0b3010 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad9211d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c06e9b0 con 0x7fad8c13dd70 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad9211d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c15f7b0 con 0x7fad8c15ea40 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c0b3010 0x7fad8c0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c0b3010 0x7fad8c0b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33370/0 (socket says 172.21.15.64:33370) 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.757+0000 7fad8b7fe640 1 -- 172.21.15.64:0/3643215125 learned_addr learned my addr 172.21.15.64:0/3643215125 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8bfff640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c15ea40 0x7fad8c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8affd640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c13dd70 0x7fad8c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8b7fe640 1 -- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c15ea40 msgr2=0x7fad8c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8b7fe640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c15ea40 0x7fad8c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8b7fe640 1 -- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c13dd70 msgr2=0x7fad8c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8b7fe640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c13dd70 0x7fad8c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8b7fe640 1 -- 172.21.15.64:0/3643215125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c153ac0 con 0x7fad8c0b3010 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad8b7fe640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c0b3010 0x7fad8c0b2690 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e760 tx=0x7fad7c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad73fff640 1 -- 172.21.15.64:0/3643215125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00f040 con 0x7fad8c0b3010 2024-03-07T07:20:23.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad73fff640 1 -- 172.21.15.64:0/3643215125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c004230 con 0x7fad8c0b3010 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad73fff640 1 -- 172.21.15.64:0/3643215125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00ee70 con 0x7fad8c0b3010 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c153da0 con 0x7fad8c0b3010 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.758+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c154360 con 0x7fad8c0b3010 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.759+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c062c90 con 0x7fad8c0b3010 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.760+0000 7fad73fff640 1 -- 172.21.15.64:0/3643215125 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fad7c00c6d0 con 0x7fad8c0b3010 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.760+0000 7fad73fff640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7806b500 0x7fad7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.760+0000 7fad73fff640 1 -- 172.21.15.64:0/3643215125 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad7c089ea0 con 0x7fad8c0b3010 2024-03-07T07:20:23.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.760+0000 7fad8affd640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7806b500 0x7fad7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:23.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.761+0000 7fad8affd640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7806b500 0x7fad7806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad80038000 tx=0x7fad80032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:23.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.765+0000 7fad73fff640 1 -- 172.21.15.64:0/3643215125 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fad7c017020 con 0x7fad8c0b3010 2024-03-07T07:20:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.946+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fad8c152720 con 0x7fad7806b500 2024-03-07T07:20:23.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.946+0000 7fad73fff640 1 -- 172.21.15.64:0/3643215125 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fad8c152720 con 0x7fad7806b500 2024-03-07T07:20:23.947 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 4 to deep-scrub 2024-03-07T07:20:23.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7806b500 msgr2=0x7fad7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7806b500 0x7fad7806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad80038000 tx=0x7fad80032000 comp rx=0 tx=0).stop 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c0b3010 msgr2=0x7fad8c0b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c0b3010 0x7fad8c0b2690 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e760 tx=0x7fad7c00ec20 comp rx=0 tx=0).stop 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 shutdown_connections 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fad7806b500 0x7fad7806d9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fad8c15ea40 0x7fad8c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fad8c13dd70 0x7fad8c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 --2- 172.21.15.64:0/3643215125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad8c0b3010 0x7fad8c0b2690 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:23.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 >> 172.21.15.64:0/3643215125 conn(0x7fad8c06b040 msgr2=0x7fad8c13b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:23.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 shutdown_connections 2024-03-07T07:20:23.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:23.948+0000 7fad9211d640 1 -- 172.21.15.64:0/3643215125 wait complete. 2024-03-07T07:20:23.962 INFO:tasks.ceph:Scrubbing osd.5 2024-03-07T07:20:23.962 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:24.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.071+0000 7fe6b64b6640 1 Processor -- start 2024-03-07T07:20:24.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.071+0000 7fe6b64b6640 1 -- start start 2024-03-07T07:20:24.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.071+0000 7fe6b64b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b00b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.071+0000 7fe6b64b6640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b015e860 con 0x7fe6b00b2e00 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.071+0000 7fe6b64b6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b015eb40 con 0x7fe6b00b31e0 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.071+0000 7fe6b64b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b015ee20 con 0x7fe6b00b6600 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6affff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7fe6b00b2e00 0x7fe6b0162750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52494/0 (socket says 172.21.15.64:52494) 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6affff640 1 -- 172.21.15.64:0/748271652 learned_addr learned my addr 172.21.15.64:0/748271652 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6b4a53640 1 --2- 172.21.15.64:0/748271652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b00b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246525945 0 0) 0x7fe6b015e860 con 0x7fe6b00b2e00 2024-03-07T07:20:24.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe698003520 con 0x7fe6b00b2e00 2024-03-07T07:20:24.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88304006 0 0) 0x7fe6b015eb40 con 0x7fe6b00b31e0 2024-03-07T07:20:24.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b015e860 con 0x7fe6b00b31e0 2024-03-07T07:20:24.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6b4a53640 1 -- 172.21.15.64:0/748271652 >> v1:172.21.15.120:6789/0 conn(0x7fe6b00b31e0 legacy=0x7fe6b00b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6b4a53640 1 -- 172.21.15.64:0/748271652 >> v1:172.21.15.64:6789/0 conn(0x7fe6b00b2e00 legacy=0x7fe6b0162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1986867192 0 0) 0x7fe698003520 con 0x7fe6b00b2e00 2024-03-07T07:20:24.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.072+0000 7fe6b4a53640 1 -- 172.21.15.64:0/748271652 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b015f100 con 0x7fe6b00b6600 2024-03-07T07:20:24.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.073+0000 7fe6b4a53640 1 --2- 172.21.15.64:0/748271652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b00b9300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009dc0 tx=0x7fe6a802bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8739bc0d21ed5902 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.073+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8039070 con 0x7fe6b00b6600 2024-03-07T07:20:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.073+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a802c880 con 0x7fe6b00b6600 2024-03-07T07:20:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.073+0000 7fe6aeffd640 1 -- 172.21.15.64:0/748271652 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0001670 con 0x7fe6b00b6600 2024-03-07T07:20:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.074+0000 7fe6b64b6640 1 -- 172.21.15.64:0/748271652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 msgr2=0x7fe6b00b9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.074+0000 7fe6b64b6640 1 --2- 172.21.15.64:0/748271652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b00b9300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009dc0 tx=0x7fe6a802bd50 comp rx=0 tx=0).stop 2024-03-07T07:20:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.074+0000 7fe6b64b6640 1 -- 172.21.15.64:0/748271652 shutdown_connections 2024-03-07T07:20:24.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.074+0000 7fe6b64b6640 1 --2- 172.21.15.64:0/748271652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b00b9300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.074+0000 7fe6b64b6640 1 -- 172.21.15.64:0/748271652 >> 172.21.15.64:0/748271652 conn(0x7fe6b00d0f50 msgr2=0x7fe6b00d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.074+0000 7fe6b64b6640 1 -- 172.21.15.64:0/748271652 shutdown_connections 2024-03-07T07:20:24.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.074+0000 7fe6b64b6640 1 -- 172.21.15.64:0/748271652 wait complete. 2024-03-07T07:20:24.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.075+0000 7fe6b64b6640 1 Processor -- start 2024-03-07T07:20:24.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.076+0000 7fe6b64b6640 1 -- start start 2024-03-07T07:20:24.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6b64b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00b2e00 0x7fe6b014e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6b64b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b00b31e0 0x7fe6b01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6b64b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b0160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6b64b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00cf0f0 con 0x7fe6b00b2e00 2024-03-07T07:20:24.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6b64b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00d3400 con 0x7fe6b00b31e0 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6b64b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00cf270 con 0x7fe6b00b6600 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00b2e00 0x7fe6b014e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00b2e00 0x7fe6b014e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33372/0 (socket says 172.21.15.64:33372) 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 -- 172.21.15.64:0/3593334235 learned_addr learned my addr 172.21.15.64:0/3593334235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6af7fe640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b00b31e0 0x7fe6b01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6b4a53640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b0160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 -- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 msgr2=0x7fe6b0160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe6b00b6600 0x7fe6b0160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 -- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b00b31e0 msgr2=0x7fe6b01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6b00b31e0 0x7fe6b01604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.077+0000 7fe6affff640 1 -- 172.21.15.64:0/3593334235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b00c9540 con 0x7fe6b00b2e00 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.078+0000 7fe6affff640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00b2e00 0x7fe6b014e580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000c520 tx=0x7fe6a000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.078+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0017070 con 0x7fe6b00b2e00 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.078+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0005d00 con 0x7fe6b00b2e00 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.078+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b00c9820 con 0x7fe6b00b2e00 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.078+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00072b0 con 0x7fe6b00b2e00 2024-03-07T07:20:24.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.078+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b00c9de0 con 0x7fe6b00b2e00 2024-03-07T07:20:24.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.078+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6b0062c90 con 0x7fe6b00b2e00 2024-03-07T07:20:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.080+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe6a001a070 con 0x7fe6b00b2e00 2024-03-07T07:20:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.080+0000 7fe6acff9640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69c06b4d0 0x7fe69c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.080+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6a00895e0 con 0x7fe6b00b2e00 2024-03-07T07:20:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.080+0000 7fe6acff9640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fe69c073670 0x7fe69c075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.080+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 --> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6a0015e70 con 0x7fe69c073670 2024-03-07T07:20:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.080+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6a0090050 con 0x7fe6b00b2e00 2024-03-07T07:20:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.080+0000 7fe6af7fe640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69c06b4d0 0x7fe69c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.081+0000 7fe6b4a53640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fe69c073670 0x7fe69c075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.081+0000 7fe6af7fe640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69c06b4d0 0x7fe69c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00ca7c0 tx=0x7fe6a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.081+0000 7fe6b4a53640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fe69c073670 0x7fe69c075a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.082+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== osd.5 v2:172.21.15.120:6800/349750424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe6a0015e70 con 0x7fe69c073670 2024-03-07T07:20:24.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.095+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 --> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe6b00b4be0 con 0x7fe69c073670 2024-03-07T07:20:24.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.095+0000 7fe6acff9640 1 -- 172.21.15.64:0/3593334235 <== osd.5 v2:172.21.15.120:6800/349750424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe6b00b4be0 con 0x7fe69c073670 2024-03-07T07:20:24.095 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:24.096 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:24.096 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.095+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fe69c073670 msgr2=0x7fe69c075a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.095+0000 7fe6b64b6640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6800/349750424,v1:172.21.15.120:6801/349750424] conn(0x7fe69c073670 0x7fe69c075a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69c06b4d0 msgr2=0x7fe69c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fe69c06b4d0 0x7fe69c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00ca7c0 tx=0x7fe6a4009040 comp rx=0 tx=0).stop 2024-03-07T07:20:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00b2e00 msgr2=0x7fe6b014e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 --2- 172.21.15.64:0/3593334235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00b2e00 0x7fe6b014e580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000c520 tx=0x7fe6a000c9e0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b4a53640 1 -- 172.21.15.64:0/3593334235 reap_dead start 2024-03-07T07:20:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 shutdown_connections 2024-03-07T07:20:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 >> 172.21.15.64:0/3593334235 conn(0x7fe6b00d0f50 msgr2=0x7fe6b015f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 shutdown_connections 2024-03-07T07:20:24.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.096+0000 7fe6b64b6640 1 -- 172.21.15.64:0/3593334235 wait complete. 2024-03-07T07:20:24.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-07T07:20:24.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.208+0000 7f5a7a283640 1 Processor -- start 2024-03-07T07:20:24.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.208+0000 7f5a7a283640 1 -- start start 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a7a283640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 0x7f5a7406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a7a283640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a740d0aa0 con 0x7f5a740b7390 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a7a283640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a740d0d80 con 0x7f5a74145bf0 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a7a283640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740d13d0 con 0x7f5a740d1860 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a78820640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f5a740b7390 0x7f5a740b5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52502/0 (socket says 172.21.15.64:52502) 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a78820640 1 -- 172.21.15.64:0/2360022323 learned_addr learned my addr 172.21.15.64:0/2360022323 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a737fe640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 0x7f5a7406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.209+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790700786 0 0) 0x7f5a740d0aa0 con 0x7f5a740b7390 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a5c003520 con 0x7f5a740b7390 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2261202248 0 0) 0x7f5a5c003520 con 0x7f5a740b7390 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 msgr2=0x7f5a7406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 0x7f5a7406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 >> v1:172.21.15.120:6789/0 conn(0x7f5a74145bf0 legacy=0x7f5a740cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740d1060 con 0x7f5a740b7390 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f5a6c0019f0 con 0x7f5a740b7390 2024-03-07T07:20:24.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 >> v1:172.21.15.64:6789/0 conn(0x7f5a740b7390 legacy=0x7f5a740b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a5c0039e0 0x7f5a5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 0x7f5a5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a5c004d40 0x7f5a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740d13d0 con 0x7f5a5c004d40 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c00b9c0 con 0x7f5a5c0039e0 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c00bbc0 con 0x7f5a740d1860 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a78820640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a5c004d40 0x7f5a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a73fff640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a5c0039e0 0x7f5a5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a737fe640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 0x7f5a5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a78820640 1 -- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 msgr2=0x7f5a5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a78820640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a740d1860 0x7f5a5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a78820640 1 -- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a5c0039e0 msgr2=0x7f5a5c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a78820640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a5c0039e0 0x7f5a5c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.210+0000 7f5a78820640 1 -- 172.21.15.64:0/2360022323 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c003520 con 0x7f5a5c004d40 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.211+0000 7f5a78820640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a5c004d40 0x7f5a5c0051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00a860 tx=0x7f5a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcfb4e8a79f4fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.211+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00a480 con 0x7f5a5c004d40 2024-03-07T07:20:24.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.211+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c034630 con 0x7f5a5c004d40 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.211+0000 7f5a72ffd640 1 -- 172.21.15.64:0/2360022323 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c034a40 con 0x7f5a5c004d40 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.212+0000 7f5a7a283640 1 -- 172.21.15.64:0/2360022323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a5c004d40 msgr2=0x7f5a5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.212+0000 7f5a7a283640 1 --2- 172.21.15.64:0/2360022323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a5c004d40 0x7f5a5c0051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00a860 tx=0x7f5a5c005830 comp rx=0 tx=0).stop 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.212+0000 7f5a78820640 1 -- 172.21.15.64:0/2360022323 reap_dead start 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.212+0000 7f5a7a283640 1 -- 172.21.15.64:0/2360022323 shutdown_connections 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.212+0000 7f5a7a283640 1 -- 172.21.15.64:0/2360022323 >> 172.21.15.64:0/2360022323 conn(0x7f5a7406d4c0 msgr2=0x7f5a74066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.212+0000 7f5a7a283640 1 -- 172.21.15.64:0/2360022323 shutdown_connections 2024-03-07T07:20:24.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.212+0000 7f5a7a283640 1 -- 172.21.15.64:0/2360022323 wait complete. 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.213+0000 7f5a7a283640 1 Processor -- start 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.213+0000 7f5a7a283640 1 -- start start 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a7415b960 0x7f5a7415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7413dcb0 0x7f5a7413e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a7413be40 0x7f5a7413c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740d0840 con 0x7f5a7413dcb0 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740d06c0 con 0x7f5a7415b960 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740b5870 con 0x7f5a7413be40 2024-03-07T07:20:24.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7413dcb0 0x7f5a7413e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7413dcb0 0x7f5a7413e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33386/0 (socket says 172.21.15.64:33386) 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 -- 172.21.15.64:0/2629065118 learned_addr learned my addr 172.21.15.64:0/2629065118 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a73fff640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a7415b960 0x7f5a7415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a78820640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a7413be40 0x7f5a7413c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 -- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a7413be40 msgr2=0x7f5a7413c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a7413be40 0x7f5a7413c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 -- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a7415b960 msgr2=0x7f5a7415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a7415b960 0x7f5a7415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 -- 172.21.15.64:0/2629065118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7415d410 con 0x7f5a7413dcb0 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a73fff640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a7415b960 0x7f5a7415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a737fe640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7413dcb0 0x7f5a7413e110 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009b60 tx=0x7f5a68015510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a70ff9640 1 -- 172.21.15.64:0/2629065118 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68020070 con 0x7f5a7413dcb0 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a70ff9640 1 -- 172.21.15.64:0/2629065118 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a68015e20 con 0x7f5a7413dcb0 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a70ff9640 1 -- 172.21.15.64:0/2629065118 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004270 con 0x7f5a7413dcb0 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7415d6f0 con 0x7f5a7413dcb0 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.215+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7415dcb0 con 0x7f5a7413dcb0 2024-03-07T07:20:24.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.216+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a74062c90 con 0x7f5a7413dcb0 2024-03-07T07:20:24.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.217+0000 7f5a70ff9640 1 -- 172.21.15.64:0/2629065118 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5a68023070 con 0x7f5a7413dcb0 2024-03-07T07:20:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.217+0000 7f5a70ff9640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5a6006b4d0 0x7f5a6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.217+0000 7f5a70ff9640 1 -- 172.21.15.64:0/2629065118 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a6808dfa0 con 0x7f5a7413dcb0 2024-03-07T07:20:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.218+0000 7f5a73fff640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5a6006b4d0 0x7f5a6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.218+0000 7f5a73fff640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5a6006b4d0 0x7f5a6006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5a64003700 tx=0x7f5a64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.222+0000 7f5a70ff9640 1 -- 172.21.15.64:0/2629065118 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a68017040 con 0x7f5a7413dcb0 2024-03-07T07:20:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.404+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a740b4b60 con 0x7f5a6006b4d0 2024-03-07T07:20:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.404+0000 7f5a70ff9640 1 -- 172.21.15.64:0/2629065118 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5a740b4b60 con 0x7f5a6006b4d0 2024-03-07T07:20:24.404 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 5 to deep-scrub 2024-03-07T07:20:24.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5a6006b4d0 msgr2=0x7f5a6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5a6006b4d0 0x7f5a6006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5a64003700 tx=0x7f5a64009040 comp rx=0 tx=0).stop 2024-03-07T07:20:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7413dcb0 msgr2=0x7f5a7413e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7413dcb0 0x7f5a7413e110 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009b60 tx=0x7f5a68015510 comp rx=0 tx=0).stop 2024-03-07T07:20:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 shutdown_connections 2024-03-07T07:20:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f5a6006b4d0 0x7f5a6006d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a7413dcb0 0x7f5a7413e110 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5a7413be40 0x7f5a7413c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 --2- 172.21.15.64:0/2629065118 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5a7415b960 0x7f5a7415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 >> 172.21.15.64:0/2629065118 conn(0x7f5a7406d4c0 msgr2=0x7f5a740b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 shutdown_connections 2024-03-07T07:20:24.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.406+0000 7f5a7a283640 1 -- 172.21.15.64:0/2629065118 wait complete. 2024-03-07T07:20:24.417 INFO:tasks.ceph:Scrubbing osd.6 2024-03-07T07:20:24.418 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:24.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.523+0000 7f3fd80c7640 1 Processor -- start 2024-03-07T07:20:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.523+0000 7f3fd80c7640 1 -- start start 2024-03-07T07:20:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.524+0000 7f3fd80c7640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fd015f350 0x7f3fd015f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.524+0000 7f3fd80c7640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00b56a0 con 0x7f3fd015bdd0 2024-03-07T07:20:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.524+0000 7f3fd80c7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd0071800 con 0x7f3fd006b510 2024-03-07T07:20:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.524+0000 7f3fd80c7640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd0071ae0 con 0x7f3fd015f350 2024-03-07T07:20:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.524+0000 7f3fd5662640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f3fd015f350 0x7f3fd015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.524+0000 7f3fd5e63640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f3fd015bdd0 0x7f3fd00b4d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52512/0 (socket says 172.21.15.64:52512) 2024-03-07T07:20:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.524+0000 7f3fd5e63640 1 -- 172.21.15.64:0/2072652125 learned_addr learned my addr 172.21.15.64:0/2072652125 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.525+0000 7f3fd5662640 1 -- 172.21.15.64:0/2072652125 >> v1:172.21.15.120:6789/0 conn(0x7f3fd006b510 legacy=0x7f3fd00722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.525+0000 7f3fd5662640 1 -- 172.21.15.64:0/2072652125 >> v1:172.21.15.64:6789/0 conn(0x7f3fd015bdd0 legacy=0x7f3fd00b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.525+0000 7f3fd4e61640 1 -- 172.21.15.64:0/2072652125 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625162698 0 0) 0x7f3fd00b56a0 con 0x7f3fd015bdd0 2024-03-07T07:20:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.525+0000 7f3fd5662640 1 -- 172.21.15.64:0/2072652125 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd015fd40 con 0x7f3fd015f350 2024-03-07T07:20:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd5662640 1 --2- 172.21.15.64:0/2072652125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015f350 0x7f3fd015f800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097a0 tx=0x7f3fc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b4c0d45cef3e66 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd4e61640 1 -- 172.21.15.64:0/2072652125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0035020 con 0x7f3fd015f350 2024-03-07T07:20:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd4e61640 1 -- 172.21.15.64:0/2072652125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0031830 con 0x7f3fd015f350 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd4e61640 1 -- 172.21.15.64:0/2072652125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0031c00 con 0x7f3fd015f350 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2072652125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015f350 msgr2=0x7f3fd015f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd80c7640 1 --2- 172.21.15.64:0/2072652125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015f350 0x7f3fd015f800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097a0 tx=0x7f3fc002b8d0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2072652125 shutdown_connections 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd80c7640 1 --2- 172.21.15.64:0/2072652125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015f350 0x7f3fd015f800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2072652125 >> 172.21.15.64:0/2072652125 conn(0x7f3fd0066e50 msgr2=0x7f3fd0067260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2072652125 shutdown_connections 2024-03-07T07:20:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.526+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2072652125 wait complete. 2024-03-07T07:20:24.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.527+0000 7f3fd80c7640 1 Processor -- start 2024-03-07T07:20:24.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.527+0000 7f3fd80c7640 1 -- start start 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd80c7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3fd006b510 0x7f3fd0155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd80c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015bdd0 0x7f3fd01f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd80c7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fd015f350 0x7f3fd01515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd80c7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0072b70 con 0x7f3fd015bdd0 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd80c7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00729f0 con 0x7f3fd015f350 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd80c7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0072cf0 con 0x7f3fd006b510 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd5662640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015bdd0 0x7f3fd01f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd5662640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015bdd0 0x7f3fd01f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33396/0 (socket says 172.21.15.64:33396) 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd5662640 1 -- 172.21.15.64:0/2952286330 learned_addr learned my addr 172.21.15.64:0/2952286330 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd6664640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fd015f350 0x7f3fd01515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd5662640 1 -- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3fd006b510 msgr2=0x7f3fd0155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd5662640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3fd006b510 0x7f3fd0155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd5662640 1 -- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fd015f350 msgr2=0x7f3fd01515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.528+0000 7f3fd5662640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fd015f350 0x7f3fd01515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fd5662640 1 -- 172.21.15.64:0/2952286330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd01525b0 con 0x7f3fd015bdd0 2024-03-07T07:20:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fd6664640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3fd015f350 0x7f3fd01515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fd5662640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015bdd0 0x7f3fd01f4600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00098c0 tx=0x7f3fc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0049070 con 0x7f3fd015bdd0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc003c9d0 con 0x7f3fd015bdd0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd0151ff0 con 0x7f3fd015bdd0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc003cdc0 con 0x7f3fd015bdd0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.529+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd01f3e20 con 0x7f3fd015bdd0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.530+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3fd0062c90 con 0x7f3fd015bdd0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.530+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3fc004c070 con 0x7f3fd015bdd0 2024-03-07T07:20:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fc67fc640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fc00ac1a0 con 0x7f3fd015bdd0 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fc67fc640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7f3fb4073670 0x7f3fb4075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 --> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fc002fe50 con 0x7f3fb4073670 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc00b3050 con 0x7f3fd015bdd0 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fd5e63640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fd6664640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7f3fb4073670 0x7f3fb4075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.531+0000 7f3fd5e63640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3fb406b4d0 0x7f3fb406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc007fd0 tx=0x7f3fcc007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.532+0000 7f3fd6664640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7f3fb4073670 0x7f3fb4075a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.532+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== osd.6 v2:172.21.15.120:6824/2639829384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3fc002fe50 con 0x7f3fb4073670 2024-03-07T07:20:24.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.545+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 --> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3fd006be20 con 0x7f3fb4073670 2024-03-07T07:20:24.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fc67fc640 1 -- 172.21.15.64:0/2952286330 <== osd.6 v2:172.21.15.120:6824/2639829384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3fd006be20 con 0x7f3fb4073670 2024-03-07T07:20:24.546 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:24.546 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:24.546 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:24.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7f3fb4073670 msgr2=0x7f3fb4075a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6824/2639829384,v1:172.21.15.120:6825/2639829384] conn(0x7f3fb4073670 0x7f3fb4075a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3fb406b4d0 msgr2=0x7f3fb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f3fb406b4d0 0x7f3fb406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc007fd0 tx=0x7f3fcc007920 comp rx=0 tx=0).stop 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015bdd0 msgr2=0x7f3fd01f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 --2- 172.21.15.64:0/2952286330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fd015bdd0 0x7f3fd01f4600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00098c0 tx=0x7f3fc00046c0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd6664640 1 -- 172.21.15.64:0/2952286330 reap_dead start 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 shutdown_connections 2024-03-07T07:20:24.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.546+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 >> 172.21.15.64:0/2952286330 conn(0x7f3fd0066e50 msgr2=0x7f3fd0159850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.547+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 shutdown_connections 2024-03-07T07:20:24.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.547+0000 7f3fd80c7640 1 -- 172.21.15.64:0/2952286330 wait complete. 2024-03-07T07:20:24.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-07T07:20:24.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.681+0000 7faac5999640 1 Processor -- start 2024-03-07T07:20:24.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.681+0000 7faac5999640 1 -- start start 2024-03-07T07:20:24.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.682+0000 7faac5999640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faac0140ec0 0x7faac01416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.682+0000 7faac5999640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac015f420 con 0x7faac015edb0 2024-03-07T07:20:24.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.682+0000 7faac5999640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00b4590 con 0x7faac0161c90 2024-03-07T07:20:24.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.682+0000 7faac5999640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac015f200 con 0x7faac0140ec0 2024-03-07T07:20:24.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.682+0000 7faabf7fe640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7faac015edb0 0x7faac00b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52518/0 (socket says 172.21.15.64:52518) 2024-03-07T07:20:24.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.682+0000 7faabf7fe640 1 -- 172.21.15.64:0/4074004906 learned_addr learned my addr 172.21.15.64:0/4074004906 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:24.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.682+0000 7faabeffd640 1 --2- 172.21.15.64:0/4074004906 >> v2:172.21.15.120:3300/0 conn(0x7faac0140ec0 0x7faac01416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.683+0000 7faabdffb640 1 -- 172.21.15.64:0/4074004906 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037445221 0 0) 0x7faac015f420 con 0x7faac015edb0 2024-03-07T07:20:24.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.683+0000 7faabdffb640 1 -- 172.21.15.64:0/4074004906 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaac003520 con 0x7faac015edb0 2024-03-07T07:20:24.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.683+0000 7faabeffd640 1 -- 172.21.15.64:0/4074004906 >> v1:172.21.15.120:6789/0 conn(0x7faac0161c90 legacy=0x7faac015e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.683+0000 7faabdffb640 1 -- 172.21.15.64:0/4074004906 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 364400206 0 0) 0x7faaac003520 con 0x7faac015edb0 2024-03-07T07:20:24.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.683+0000 7faabeffd640 1 -- 172.21.15.64:0/4074004906 >> v1:172.21.15.64:6789/0 conn(0x7faac015edb0 legacy=0x7faac00b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.683+0000 7faabeffd640 1 -- 172.21.15.64:0/4074004906 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac00b4870 con 0x7faac0140ec0 2024-03-07T07:20:24.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faabeffd640 1 --2- 172.21.15.64:0/4074004906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 0x7faac01416f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faab80097a0 tx=0x7faab802b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2606aede69a7d8d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faabdffb640 1 -- 172.21.15.64:0/4074004906 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8037070 con 0x7faac0140ec0 2024-03-07T07:20:24.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faabdffb640 1 -- 172.21.15.64:0/4074004906 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8030510 con 0x7faac0140ec0 2024-03-07T07:20:24.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faabdffb640 1 -- 172.21.15.64:0/4074004906 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab00019f0 con 0x7faac0140ec0 2024-03-07T07:20:24.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faac5999640 1 -- 172.21.15.64:0/4074004906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 msgr2=0x7faac01416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faac5999640 1 --2- 172.21.15.64:0/4074004906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 0x7faac01416f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faab80097a0 tx=0x7faab802b680 comp rx=0 tx=0).stop 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faac5999640 1 -- 172.21.15.64:0/4074004906 shutdown_connections 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faac5999640 1 --2- 172.21.15.64:0/4074004906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 0x7faac01416f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faac5999640 1 -- 172.21.15.64:0/4074004906 >> 172.21.15.64:0/4074004906 conn(0x7faac0071340 msgr2=0x7faac0071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faac5999640 1 -- 172.21.15.64:0/4074004906 shutdown_connections 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.684+0000 7faac5999640 1 -- 172.21.15.64:0/4074004906 wait complete. 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 Processor -- start 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 -- start start 2024-03-07T07:20:24.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 0x7faac00d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac015edb0 0x7faac0142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faac0161c90 0x7faac0154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac0141420 con 0x7faac015edb0 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01412a0 con 0x7faac0140ec0 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.686+0000 7faac5999640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac0162110 con 0x7faac0161c90 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac015edb0 0x7faac0142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac015edb0 0x7faac0142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33406/0 (socket says 172.21.15.64:33406) 2024-03-07T07:20:24.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 -- 172.21.15.64:0/1288178314 learned_addr learned my addr 172.21.15.64:0/1288178314 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabeffd640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 0x7faac00d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabf7fe640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faac0161c90 0x7faac0154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 -- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faac0161c90 msgr2=0x7faac0154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faac0161c90 0x7faac0154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 -- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 msgr2=0x7faac00d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 0x7faac00d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 -- 172.21.15.64:0/1288178314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0143510 con 0x7faac015edb0 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faabe7fc640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac015edb0 0x7faac0142fd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c780 tx=0x7faaa800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faab7fff640 1 -- 172.21.15.64:0/1288178314 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8017070 con 0x7faac015edb0 2024-03-07T07:20:24.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faab7fff640 1 -- 172.21.15.64:0/1288178314 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8005290 con 0x7faac015edb0 2024-03-07T07:20:24.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac01437e0 con 0x7faac015edb0 2024-03-07T07:20:24.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faab7fff640 1 -- 172.21.15.64:0/1288178314 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800a880 con 0x7faac015edb0 2024-03-07T07:20:24.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.687+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac0143da0 con 0x7faac015edb0 2024-03-07T07:20:24.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.688+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac00d1d30 con 0x7faac015edb0 2024-03-07T07:20:24.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.689+0000 7faab7fff640 1 -- 172.21.15.64:0/1288178314 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faaa801a070 con 0x7faac015edb0 2024-03-07T07:20:24.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.689+0000 7faab7fff640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7faaa006b4d0 0x7faaa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.689+0000 7faab7fff640 1 -- 172.21.15.64:0/1288178314 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faaa8089740 con 0x7faac015edb0 2024-03-07T07:20:24.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.689+0000 7faabeffd640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7faaa006b4d0 0x7faaa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.690+0000 7faabeffd640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7faaa006b4d0 0x7faaa006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faab8034000 tx=0x7faab8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.693+0000 7faab7fff640 1 -- 172.21.15.64:0/1288178314 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faaa8058540 con 0x7faac015edb0 2024-03-07T07:20:24.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.874+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7faac0062c90 con 0x7faaa006b4d0 2024-03-07T07:20:24.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.874+0000 7faab7fff640 1 -- 172.21.15.64:0/1288178314 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faac0062c90 con 0x7faaa006b4d0 2024-03-07T07:20:24.875 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 6 to deep-scrub 2024-03-07T07:20:24.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7faaa006b4d0 msgr2=0x7faaa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7faaa006b4d0 0x7faaa006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faab8034000 tx=0x7faab8008030 comp rx=0 tx=0).stop 2024-03-07T07:20:24.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac015edb0 msgr2=0x7faac0142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac015edb0 0x7faac0142fd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c780 tx=0x7faaa800cc40 comp rx=0 tx=0).stop 2024-03-07T07:20:24.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 shutdown_connections 2024-03-07T07:20:24.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7faaa006b4d0 0x7faaa006d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faac015edb0 0x7faac0142fd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faac0161c90 0x7faac0154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 --2- 172.21.15.64:0/1288178314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faac0140ec0 0x7faac00d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 >> 172.21.15.64:0/1288178314 conn(0x7faac0071340 msgr2=0x7faac01573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:24.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 shutdown_connections 2024-03-07T07:20:24.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.876+0000 7faac5999640 1 -- 172.21.15.64:0/1288178314 wait complete. 2024-03-07T07:20:24.887 INFO:tasks.ceph:Scrubbing osd.7 2024-03-07T07:20:24.887 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-07T07:20:24.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.993+0000 7fa2d7beb640 1 Processor -- start 2024-03-07T07:20:24.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.993+0000 7fa2d7beb640 1 -- start start 2024-03-07T07:20:24.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d7beb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e210 0x7fa2d00b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d7beb640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa2d015e5f0 0x7fa2d0161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:24.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d7beb640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d0162880 con 0x7fa2d0162460 2024-03-07T07:20:24.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d7beb640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d00d2640 con 0x7fa2d015e5f0 2024-03-07T07:20:24.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d7beb640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d0071880 con 0x7fa2d015e210 2024-03-07T07:20:24.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5186640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa2d015e5f0 0x7fa2d0161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5987640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e210 0x7fa2d00b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:24.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5186640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa2d015e5f0 0x7fa2d0161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:53138/0 (socket says 172.21.15.64:53138) 2024-03-07T07:20:24.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5186640 1 -- 172.21.15.64:0/393044646 learned_addr learned my addr 172.21.15.64:0/393044646 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:24.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5987640 1 -- 172.21.15.64:0/393044646 >> v2:172.21.15.120:3300/0 conn(0x7fa2d015e5f0 msgr2=0x7fa2d0161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:24.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5987640 1 --2- 172.21.15.64:0/393044646 >> v2:172.21.15.120:3300/0 conn(0x7fa2d015e5f0 0x7fa2d0161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:24.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5987640 1 -- 172.21.15.64:0/393044646 >> v1:172.21.15.120:6789/0 conn(0x7fa2d0162460 legacy=0x7fa2d0070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:24.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5987640 1 -- 172.21.15.64:0/393044646 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0071b60 con 0x7fa2d015e210 2024-03-07T07:20:24.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.994+0000 7fa2d5186640 1 --2- 172.21.15.64:0/393044646 >> v2:172.21.15.120:3300/0 conn(0x7fa2d015e5f0 0x7fa2d0161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:24.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.995+0000 7fa2d5987640 1 --2- 172.21.15.64:0/393044646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e210 0x7fa2d00b9a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0097a0 tx=0x7fa2cc02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936f33e085b3432b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:24.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.995+0000 7fa2d4985640 1 -- 172.21.15.64:0/393044646 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc035020 con 0x7fa2d015e210 2024-03-07T07:20:24.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.995+0000 7fa2d4985640 1 -- 172.21.15.64:0/393044646 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc031810 con 0x7fa2d015e210 2024-03-07T07:20:24.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.995+0000 7fa2d4985640 1 -- 172.21.15.64:0/393044646 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc031be0 con 0x7fa2d015e210 2024-03-07T07:20:24.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 -- 172.21.15.64:0/393044646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e210 msgr2=0x7fa2d00b9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 --2- 172.21.15.64:0/393044646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e210 0x7fa2d00b9a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0097a0 tx=0x7fa2cc02b8b0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 -- 172.21.15.64:0/393044646 shutdown_connections 2024-03-07T07:20:25.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 --2- 172.21.15.64:0/393044646 >> v2:172.21.15.120:3300/0 conn(0x7fa2d015e5f0 0x7fa2d0161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 --2- 172.21.15.64:0/393044646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e210 0x7fa2d00b9a40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 -- 172.21.15.64:0/393044646 >> 172.21.15.64:0/393044646 conn(0x7fa2d00d40b0 msgr2=0x7fa2d00d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:25.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 -- 172.21.15.64:0/393044646 shutdown_connections 2024-03-07T07:20:25.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.996+0000 7fa2d7beb640 1 -- 172.21.15.64:0/393044646 wait complete. 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.997+0000 7fa2d7beb640 1 Processor -- start 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.997+0000 7fa2d7beb640 1 -- start start 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d7beb640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2d015e210 0x7fa2d014f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d7beb640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e5f0 0x7fa2d00ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d7beb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2d0162460 0x7fa2d00ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d7beb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d00b9f80 con 0x7fa2d0162460 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d7beb640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d0071e40 con 0x7fa2d015e210 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d7beb640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d00d3ae0 con 0x7fa2d015e5f0 2024-03-07T07:20:25.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2d0162460 0x7fa2d00ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d5987640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2d015e210 0x7fa2d014f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2d0162460 0x7fa2d00ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33420/0 (socket says 172.21.15.64:33420) 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d5987640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2d015e210 0x7fa2d014f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:53154/0 (socket says 172.21.15.64:53154) 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 -- 172.21.15.64:0/1716673237 learned_addr learned my addr 172.21.15.64:0/1716673237 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d5186640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e5f0 0x7fa2d00ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 -- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e5f0 msgr2=0x7fa2d00ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa2d015e5f0 0x7fa2d00ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 -- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2d015e210 msgr2=0x7fa2d014f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa2d015e210 0x7fa2d014f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 -- 172.21.15.64:0/1716673237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00cd870 con 0x7fa2d0162460 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.998+0000 7fa2d6188640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2d0162460 0x7fa2d00ac9f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c6b0 tx=0x7fa2c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.999+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8003b70 con 0x7fa2d0162460 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.999+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8003d30 con 0x7fa2d0162460 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.999+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005310 con 0x7fa2d0162460 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.999+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d00cbda0 con 0x7fa2d0162460 2024-03-07T07:20:25.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:24.999+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d00cd4b0 con 0x7fa2d0162460 2024-03-07T07:20:25.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.000+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2d00cbb90 con 0x7fa2d0162460 2024-03-07T07:20:25.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.000+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa2c80054b0 con 0x7fa2d0162460 2024-03-07T07:20:25.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.000+0000 7fa2c67fc640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa2b406b4d0 0x7fa2b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.000+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2c8089fc0 con 0x7fa2d0162460 2024-03-07T07:20:25.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.000+0000 7fa2c67fc640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7fa2b4073670 0x7fa2b4075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.001+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 --> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2c801de70 con 0x7fa2b4073670 2024-03-07T07:20:25.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.001+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c808c5e0 con 0x7fa2d0162460 2024-03-07T07:20:25.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.001+0000 7fa2d5987640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa2b406b4d0 0x7fa2b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.001+0000 7fa2d5186640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7fa2b4073670 0x7fa2b4075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.001+0000 7fa2d5987640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa2b406b4d0 0x7fa2b406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc036000 tx=0x7fa2cc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.001+0000 7fa2d5186640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7fa2b4073670 0x7fa2b4075a90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.002+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== osd.7 v2:172.21.15.120:6816/1578261849 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa2c801de70 con 0x7fa2b4073670 2024-03-07T07:20:25.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.028+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 --> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa2d0062c90 con 0x7fa2b4073670 2024-03-07T07:20:25.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.028+0000 7fa2c67fc640 1 -- 172.21.15.64:0/1716673237 <== osd.7 v2:172.21.15.120:6816/1578261849 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa2d0062c90 con 0x7fa2b4073670 2024-03-07T07:20:25.032 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-03-07T07:20:25.032 INFO:teuthology.orchestra.run.smithi064.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T07:20:25.032 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-03-07T07:20:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7fa2b4073670 msgr2=0x7fa2b4075a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6816/1578261849,v1:172.21.15.120:6817/1578261849] conn(0x7fa2b4073670 0x7fa2b4075a90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa2b406b4d0 msgr2=0x7fa2b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fa2b406b4d0 0x7fa2b406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc036000 tx=0x7fa2cc008030 comp rx=0 tx=0).stop 2024-03-07T07:20:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2d0162460 msgr2=0x7fa2d00ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 --2- 172.21.15.64:0/1716673237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2d0162460 0x7fa2d00ac9f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c6b0 tx=0x7fa2c800cb70 comp rx=0 tx=0).stop 2024-03-07T07:20:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d6188640 1 -- 172.21.15.64:0/1716673237 reap_dead start 2024-03-07T07:20:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 shutdown_connections 2024-03-07T07:20:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 >> 172.21.15.64:0/1716673237 conn(0x7fa2d00d40b0 msgr2=0x7fa2d00b5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 shutdown_connections 2024-03-07T07:20:25.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.029+0000 7fa2d7beb640 1 -- 172.21.15.64:0/1716673237 wait complete. 2024-03-07T07:20:25.039 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-07T07:20:25.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.145+0000 7f1086234640 1 Processor -- start 2024-03-07T07:20:25.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.145+0000 7f1086234640 1 -- start start 2024-03-07T07:20:25.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f1086234640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 0x7f1080162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f1086234640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10800b4bd0 con 0x7f1080162ab0 2024-03-07T07:20:25.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f1086234640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10800b4eb0 con 0x7f10800b5dd0 2024-03-07T07:20:25.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f1086234640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1080141700 con 0x7f1080144540 2024-03-07T07:20:25.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7f1080162ab0 0x7f1080071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52522/0 (socket says 172.21.15.64:52522) 2024-03-07T07:20:25.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107ffff640 1 -- 172.21.15.64:0/2205146872 learned_addr learned my addr 172.21.15.64:0/2205146872 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:25.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107effd640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 0x7f1080162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784906656 0 0) 0x7f10800b4bd0 con 0x7f1080162ab0 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1068003520 con 0x7f1080162ab0 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595822402 0 0) 0x7f10800b4eb0 con 0x7f10800b5dd0 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10800b4bd0 con 0x7f10800b5dd0 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.146+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1111660512 0 0) 0x7f1068003520 con 0x7f1080162ab0 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 msgr2=0x7f1080162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 0x7f1080162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 >> v1:172.21.15.120:6789/0 conn(0x7f10800b5dd0 legacy=0x7f10800b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10800b5190 con 0x7f1080162ab0 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7f10780019f0 con 0x7f1080162ab0 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107effd640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 0x7f1080162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 >> v1:172.21.15.64:6789/0 conn(0x7f1080162ab0 legacy=0x7f1080071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1068003b70 0x7f1068003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 0x7f1068004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1068004d40 0x7f10680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1080141700 con 0x7f1068004d40 2024-03-07T07:20:25.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106800baa0 con 0x7f1068003b70 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106800bca0 con 0x7f1080144540 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107ffff640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1068004d40 0x7f10680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107f7fe640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1068003b70 0x7f1068003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107effd640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 0x7f1068004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107ffff640 1 -- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 msgr2=0x7f1068004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107ffff640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1080144540 0x7f1068004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107ffff640 1 -- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1068003b70 msgr2=0x7f1068003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:25.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107ffff640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1068003b70 0x7f1068003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.147+0000 7f107ffff640 1 -- 172.21.15.64:0/2205146872 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068003520 con 0x7f1068004d40 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.148+0000 7f107ffff640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1068004d40 0x7f10680051c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10800722a0 tx=0x7f1068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6f99b071ebd86 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.148+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107800a410 con 0x7f1068004d40 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.148+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1078034650 con 0x7f1068004d40 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.148+0000 7f107e7fc640 1 -- 172.21.15.64:0/2205146872 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078034a40 con 0x7f1068004d40 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.149+0000 7f1086234640 1 -- 172.21.15.64:0/2205146872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1068004d40 msgr2=0x7f10680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.149+0000 7f1086234640 1 --2- 172.21.15.64:0/2205146872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1068004d40 0x7f10680051c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10800722a0 tx=0x7f1068005830 comp rx=0 tx=0).stop 2024-03-07T07:20:25.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.149+0000 7f107ffff640 1 -- 172.21.15.64:0/2205146872 reap_dead start 2024-03-07T07:20:25.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.149+0000 7f1086234640 1 -- 172.21.15.64:0/2205146872 shutdown_connections 2024-03-07T07:20:25.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.149+0000 7f1086234640 1 -- 172.21.15.64:0/2205146872 >> 172.21.15.64:0/2205146872 conn(0x7f10800d0e70 msgr2=0x7f10800d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:25.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.149+0000 7f1086234640 1 -- 172.21.15.64:0/2205146872 shutdown_connections 2024-03-07T07:20:25.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.149+0000 7f1086234640 1 -- 172.21.15.64:0/2205146872 wait complete. 2024-03-07T07:20:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.150+0000 7f1086234640 1 Processor -- start 2024-03-07T07:20:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.151+0000 7f1086234640 1 -- start start 2024-03-07T07:20:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f1086234640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f108013cd00 0x7f1080159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f1086234640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1080157c40 0x7f1080159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f1086234640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080158190 0x7f108013d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f1086234640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10800b61b0 con 0x7f1080158190 2024-03-07T07:20:25.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f1086234640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1080161e00 con 0x7f1080157c40 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f1086234640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1080071b20 con 0x7f108013cd00 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080158190 0x7f108013d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080158190 0x7f108013d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33424/0 (socket says 172.21.15.64:33424) 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 -- 172.21.15.64:0/2186120879 learned_addr learned my addr 172.21.15.64:0/2186120879 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107effd640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1080157c40 0x7f1080159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107f7fe640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f108013cd00 0x7f1080159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 -- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f108013cd00 msgr2=0x7f1080159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f108013cd00 0x7f1080159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 -- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1080157c40 msgr2=0x7f1080159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1080157c40 0x7f1080159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107ffff640 1 -- 172.21.15.64:0/2186120879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108015bd90 con 0x7f1080158190 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.152+0000 7f107f7fe640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f108013cd00 0x7f1080159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.153+0000 7f107ffff640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080158190 0x7f108013d490 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1078001fd0 tx=0x7f10800b5080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.153+0000 7f1067fff640 1 -- 172.21.15.64:0/2186120879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078044070 con 0x7f1080158190 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.153+0000 7f1067fff640 1 -- 172.21.15.64:0/2186120879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107803a040 con 0x7f1080158190 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.153+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108015b4f0 con 0x7f1080158190 2024-03-07T07:20:25.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.153+0000 7f1067fff640 1 -- 172.21.15.64:0/2186120879 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078004b60 con 0x7f1080158190 2024-03-07T07:20:25.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.153+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108015bab0 con 0x7f1080158190 2024-03-07T07:20:25.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.154+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108015f0d0 con 0x7f1080158190 2024-03-07T07:20:25.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.155+0000 7f1067fff640 1 -- 172.21.15.64:0/2186120879 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1078047070 con 0x7f1080158190 2024-03-07T07:20:25.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.155+0000 7f1067fff640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f106c06b500 0x7f106c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.155+0000 7f1067fff640 1 -- 172.21.15.64:0/2186120879 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10780ab860 con 0x7f1080158190 2024-03-07T07:20:25.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.155+0000 7f107f7fe640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f106c06b500 0x7f106c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.156+0000 7f107f7fe640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f106c06b500 0x7f106c06d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f10700055e0 tx=0x7f1070009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.160+0000 7f1067fff640 1 -- 172.21.15.64:0/2186120879 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f107807a650 con 0x7f1080158190 2024-03-07T07:20:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.341+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f1080062c90 con 0x7f106c06b500 2024-03-07T07:20:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.341+0000 7f1067fff640 1 -- 172.21.15.64:0/2186120879 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1080062c90 con 0x7f106c06b500 2024-03-07T07:20:25.341 INFO:teuthology.orchestra.run.smithi064.stderr:instructed osd(s) 7 to deep-scrub 2024-03-07T07:20:25.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f106c06b500 msgr2=0x7f106c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f106c06b500 0x7f106c06d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f10700055e0 tx=0x7f1070009040 comp rx=0 tx=0).stop 2024-03-07T07:20:25.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080158190 msgr2=0x7f108013d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080158190 0x7f108013d490 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1078001fd0 tx=0x7f10800b5080 comp rx=0 tx=0).stop 2024-03-07T07:20:25.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 shutdown_connections 2024-03-07T07:20:25.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f106c06b500 0x7f106c06d9c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1080158190 0x7f108013d490 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1080157c40 0x7f1080159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 --2- 172.21.15.64:0/2186120879 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f108013cd00 0x7f1080159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.343+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 >> 172.21.15.64:0/2186120879 conn(0x7f10800d0e70 msgr2=0x7f10800b7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:25.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.344+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 shutdown_connections 2024-03-07T07:20:25.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.344+0000 7f1086234640 1 -- 172.21.15.64:0/2186120879 wait complete. 2024-03-07T07:20:25.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:20:25.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.459+0000 7ffa764dd640 1 Processor -- start 2024-03-07T07:20:25.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.459+0000 7ffa764dd640 1 -- start start 2024-03-07T07:20:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa764dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 0x7ffa7006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa764dd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ffa700b7390 0x7ffa7006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa764dd640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700d0aa0 con 0x7ffa70145bf0 2024-03-07T07:20:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa764dd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700d13d0 con 0x7ffa700b7390 2024-03-07T07:20:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa764dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d0d80 con 0x7ffa700d1860 2024-03-07T07:20:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6ffff640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ffa70145bf0 0x7ffa700cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:52536/0 (socket says 172.21.15.64:52536) 2024-03-07T07:20:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6ffff640 1 -- 172.21.15.64:0/339153270 learned_addr learned my addr 172.21.15.64:0/339153270 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa74a7a640 1 --2- 172.21.15.64:0/339153270 >> v2:172.21.15.120:3300/0 conn(0x7ffa700b7390 0x7ffa7006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6f7fe640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 0x7ffa7006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135297478 0 0) 0x7ffa700d0aa0 con 0x7ffa70145bf0 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58003520 con 0x7ffa70145bf0 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833562877 0 0) 0x7ffa58003520 con 0x7ffa70145bf0 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 msgr2=0x7ffa7006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 0x7ffa7006b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 >> v2:172.21.15.120:3300/0 conn(0x7ffa700b7390 msgr2=0x7ffa7006ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 --2- 172.21.15.64:0/339153270 >> v2:172.21.15.120:3300/0 conn(0x7ffa700b7390 0x7ffa7006ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 --> v1:172.21.15.64:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700d1060 con 0x7ffa70145bf0 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 <== mon.0 v1:172.21.15.64:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1521107589 0 0) 0x7ffa60001670 con 0x7ffa70145bf0 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 >> v1:172.21.15.64:6789/0 conn(0x7ffa70145bf0 legacy=0x7ffa700cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 0x7ffa580039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58003f50 0x7ffa58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa58004d40 0x7ffa580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d13d0 con 0x7ffa58004d40 2024-03-07T07:20:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d0d80 con 0x7ffa58003f50 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.460+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800b9c0 con 0x7ffa700d1860 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa74a7a640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa58004d40 0x7ffa580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa6ffff640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 0x7ffa580039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa6f7fe640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58003f50 0x7ffa58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa74a7a640 1 -- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 msgr2=0x7ffa580039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa74a7a640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa700d1860 0x7ffa580039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa74a7a640 1 -- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58003f50 msgr2=0x7ffa58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa74a7a640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58003f50 0x7ffa58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.461+0000 7ffa74a7a640 1 -- 172.21.15.64:0/339153270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58003520 con 0x7ffa58004d40 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa74a7a640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa58004d40 0x7ffa580051c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800e170 tx=0x7ffa58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce0a781eea9a981 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68039070 con 0x7ffa58004d40 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6802cc20 con 0x7ffa58004d40 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa6effd640 1 -- 172.21.15.64:0/339153270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68009c10 con 0x7ffa58004d40 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa764dd640 1 -- 172.21.15.64:0/339153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa58004d40 msgr2=0x7ffa580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa764dd640 1 --2- 172.21.15.64:0/339153270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa58004d40 0x7ffa580051c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800e170 tx=0x7ffa58005830 comp rx=0 tx=0).stop 2024-03-07T07:20:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa74a7a640 1 -- 172.21.15.64:0/339153270 reap_dead start 2024-03-07T07:20:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa764dd640 1 -- 172.21.15.64:0/339153270 shutdown_connections 2024-03-07T07:20:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa764dd640 1 -- 172.21.15.64:0/339153270 >> 172.21.15.64:0/339153270 conn(0x7ffa7006d4c0 msgr2=0x7ffa70066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa764dd640 1 -- 172.21.15.64:0/339153270 shutdown_connections 2024-03-07T07:20:25.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.462+0000 7ffa764dd640 1 -- 172.21.15.64:0/339153270 wait complete. 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.463+0000 7ffa764dd640 1 Processor -- start 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa764dd640 1 -- start start 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa764dd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58004d40 0x7ffa701f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa764dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa70071550 0x7ffa7015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa764dd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa70072b40 0x7ffa700cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa764dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d0840 con 0x7ffa70071550 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa764dd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d06c0 con 0x7ffa58004d40 2024-03-07T07:20:25.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa764dd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7006b510 con 0x7ffa70072b40 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.464+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa70071550 0x7ffa7015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa70071550 0x7ffa7015bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33432/0 (socket says 172.21.15.64:33432) 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 -- 172.21.15.64:0/2209717541 learned_addr learned my addr 172.21.15.64:0/2209717541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6ffff640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58004d40 0x7ffa701f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa74a7a640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa70072b40 0x7ffa700cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 -- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa70072b40 msgr2=0x7ffa700cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa70072b40 0x7ffa700cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 -- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58004d40 msgr2=0x7ffa701f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58004d40 0x7ffa701f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 -- 172.21.15.64:0/2209717541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700ce0c0 con 0x7ffa70071550 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa74a7a640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa70072b40 0x7ffa700cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6ffff640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58004d40 0x7ffa701f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6f7fe640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa70071550 0x7ffa7015bda0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffa64011960 tx=0x7ffa64011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6cff9640 1 -- 172.21.15.64:0/2209717541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64019070 con 0x7ffa70071550 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6cff9640 1 -- 172.21.15.64:0/2209717541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64004240 con 0x7ffa70071550 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa70071aa0 con 0x7ffa70071550 2024-03-07T07:20:25.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa6cff9640 1 -- 172.21.15.64:0/2209717541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400f7c0 con 0x7ffa70071550 2024-03-07T07:20:25.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.465+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa70072060 con 0x7ffa70071550 2024-03-07T07:20:25.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.466+0000 7ffa6cff9640 1 -- 172.21.15.64:0/2209717541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffa6401c070 con 0x7ffa70071550 2024-03-07T07:20:25.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.466+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa70062c90 con 0x7ffa70071550 2024-03-07T07:20:25.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.467+0000 7ffa6cff9640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:25.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.467+0000 7ffa6cff9640 1 -- 172.21.15.64:0/2209717541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa6408a620 con 0x7ffa70071550 2024-03-07T07:20:25.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.467+0000 7ffa6ffff640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:25.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.468+0000 7ffa6ffff640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa70073060 tx=0x7ffa60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:25.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.471+0000 7ffa6cff9640 1 -- 172.21.15.64:0/2209717541 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffa700ce0c0 con 0x7ffa70071550 2024-03-07T07:20:25.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.652+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa700b4b60 con 0x7ffa5c06b4d0 2024-03-07T07:20:25.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.658+0000 7ffa6cff9640 1 -- 172.21.15.64:0/2209717541 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251840 (secure 0 0 0) 0x7ffa700b4b60 con 0x7ffa5c06b4d0 2024-03-07T07:20:25.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:25.662 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:20:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ffa5c06b4d0 msgr2=0x7ffa5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa70073060 tx=0x7ffa60009040 comp rx=0 tx=0).stop 2024-03-07T07:20:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa70071550 msgr2=0x7ffa7015bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa70071550 0x7ffa7015bda0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffa64011960 tx=0x7ffa64011e20 comp rx=0 tx=0).stop 2024-03-07T07:20:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 shutdown_connections 2024-03-07T07:20:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ffa5c06b4d0 0x7ffa5c06d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa70071550 0x7ffa7015bda0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa70072b40 0x7ffa700cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 --2- 172.21.15.64:0/2209717541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa58004d40 0x7ffa701f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 >> 172.21.15.64:0/2209717541 conn(0x7ffa7006d4c0 msgr2=0x7ffa70066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 shutdown_connections 2024-03-07T07:20:25.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:25.662+0000 7ffa764dd640 1 -- 172.21.15.64:0/2209717541 wait complete. 2024-03-07T07:20:25.673 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-03-07T07:20:24.193180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598311,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220308,"kb_used_data":4144,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754412,"statfs":{"total":773094113280,"available":772868517888,"internally_reserved":0,"allocated":4243456,"data_stored":2369550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5533,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002740"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018002+0000","last_change":"2024-03-07T07:20:08.025425+0000","last_active":"2024-03-07T07:20:09.018002+0000","last_peered":"2024-03-07T07:20:09.018002+0000","last_clean":"2024-03-07T07:20:09.018002+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:09.018002+0000","last_undegraded":"2024-03-07T07:20:09.018002+0000","last_fullsized":"2024-03-07T07:20:09.018002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:16.203724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516922+0000","last_change":"2024-03-07T07:20:08.036714+0000","last_active":"2024-03-07T07:20:09.516922+0000","last_peered":"2024-03-07T07:20:09.516922+0000","last_clean":"2024-03-07T07:20:09.516922+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:09.516922+0000","last_undegraded":"2024-03-07T07:20:09.516922+0000","last_fullsized":"2024-03-07T07:20:09.516922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:43.043942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576955+0000","last_change":"2024-03-07T07:20:08.037973+0000","last_active":"2024-03-07T07:20:09.576955+0000","last_peered":"2024-03-07T07:20:09.576955+0000","last_clean":"2024-03-07T07:20:09.576955+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:09.576955+0000","last_undegraded":"2024-03-07T07:20:09.576955+0000","last_fullsized":"2024-03-07T07:20:09.576955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575962+0000","last_change":"2024-03-07T07:20:08.042993+0000","last_active":"2024-03-07T07:20:09.575962+0000","last_peered":"2024-03-07T07:20:09.575962+0000","last_clean":"2024-03-07T07:20:09.575962+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:09.575962+0000","last_undegraded":"2024-03-07T07:20:09.575962+0000","last_fullsized":"2024-03-07T07:20:09.575962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:47:55.462989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575521+0000","last_change":"2024-03-07T07:20:08.037872+0000","last_active":"2024-03-07T07:20:09.575521+0000","last_peered":"2024-03-07T07:20:09.575521+0000","last_clean":"2024-03-07T07:20:09.575521+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:09.575521+0000","last_undegraded":"2024-03-07T07:20:09.575521+0000","last_fullsized":"2024-03-07T07:20:09.575521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:47.883124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516617+0000","last_change":"2024-03-07T07:20:08.028212+0000","last_active":"2024-03-07T07:20:09.516617+0000","last_peered":"2024-03-07T07:20:09.516617+0000","last_clean":"2024-03-07T07:20:09.516617+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:09.516617+0000","last_undegraded":"2024-03-07T07:20:09.516617+0000","last_fullsized":"2024-03-07T07:20:09.516617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577021+0000","last_change":"2024-03-07T07:20:08.032379+0000","last_active":"2024-03-07T07:20:09.577021+0000","last_peered":"2024-03-07T07:20:09.577021+0000","last_clean":"2024-03-07T07:20:09.577021+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:09.577021+0000","last_undegraded":"2024-03-07T07:20:09.577021+0000","last_fullsized":"2024-03-07T07:20:09.577021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576904+0000","last_change":"2024-03-07T07:20:08.035495+0000","last_active":"2024-03-07T07:20:09.576904+0000","last_peered":"2024-03-07T07:20:09.576904+0000","last_clean":"2024-03-07T07:20:09.576904+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:09.576904+0000","last_undegraded":"2024-03-07T07:20:09.576904+0000","last_fullsized":"2024-03-07T07:20:09.576904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018731+0000","last_change":"2024-03-07T07:20:08.034012+0000","last_active":"2024-03-07T07:20:09.018731+0000","last_peered":"2024-03-07T07:20:09.018731+0000","last_clean":"2024-03-07T07:20:09.018731+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:09.018731+0000","last_undegraded":"2024-03-07T07:20:09.018731+0000","last_fullsized":"2024-03-07T07:20:09.018731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:55:42.535833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576070+0000","last_change":"2024-03-07T07:20:08.037279+0000","last_active":"2024-03-07T07:20:09.576070+0000","last_peered":"2024-03-07T07:20:09.576070+0000","last_clean":"2024-03-07T07:20:09.576070+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:09.576070+0000","last_undegraded":"2024-03-07T07:20:09.576070+0000","last_fullsized":"2024-03-07T07:20:09.576070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:15.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576985+0000","last_change":"2024-03-07T07:20:08.039682+0000","last_active":"2024-03-07T07:20:09.576985+0000","last_peered":"2024-03-07T07:20:09.576985+0000","last_clean":"2024-03-07T07:20:09.576985+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:09.576985+0000","last_undegraded":"2024-03-07T07:20:09.576985+0000","last_fullsized":"2024-03-07T07:20:09.576985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576401+0000","last_change":"2024-03-07T07:20:08.028305+0000","last_active":"2024-03-07T07:20:09.576401+0000","last_peered":"2024-03-07T07:20:09.576401+0000","last_clean":"2024-03-07T07:20:09.576401+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:09.576401+0000","last_undegraded":"2024-03-07T07:20:09.576401+0000","last_fullsized":"2024-03-07T07:20:09.576401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575664+0000","last_change":"2024-03-07T07:20:08.033602+0000","last_active":"2024-03-07T07:20:09.575664+0000","last_peered":"2024-03-07T07:20:09.575664+0000","last_clean":"2024-03-07T07:20:09.575664+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:09.575664+0000","last_undegraded":"2024-03-07T07:20:09.575664+0000","last_fullsized":"2024-03-07T07:20:09.575664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:55.373070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516717+0000","last_change":"2024-03-07T07:20:08.022351+0000","last_active":"2024-03-07T07:20:09.516717+0000","last_peered":"2024-03-07T07:20:09.516717+0000","last_clean":"2024-03-07T07:20:09.516717+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:09.516717+0000","last_undegraded":"2024-03-07T07:20:09.516717+0000","last_fullsized":"2024-03-07T07:20:09.516717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:44.173378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:20:08.021148+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577186+0000","last_change":"2024-03-07T07:20:08.035791+0000","last_active":"2024-03-07T07:20:09.577186+0000","last_peered":"2024-03-07T07:20:09.577186+0000","last_clean":"2024-03-07T07:20:09.577186+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:09.577186+0000","last_undegraded":"2024-03-07T07:20:09.577186+0000","last_fullsized":"2024-03-07T07:20:09.577186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576963+0000","last_change":"2024-03-07T07:20:08.034016+0000","last_active":"2024-03-07T07:20:09.576963+0000","last_peered":"2024-03-07T07:20:09.576963+0000","last_clean":"2024-03-07T07:20:09.576963+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:09.576963+0000","last_undegraded":"2024-03-07T07:20:09.576963+0000","last_fullsized":"2024-03-07T07:20:09.576963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517000+0000","last_change":"2024-03-07T07:20:08.035489+0000","last_active":"2024-03-07T07:20:09.517000+0000","last_peered":"2024-03-07T07:20:09.517000+0000","last_clean":"2024-03-07T07:20:09.517000+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:09.517000+0000","last_undegraded":"2024-03-07T07:20:09.517000+0000","last_fullsized":"2024-03-07T07:20:09.517000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:07.455855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516748+0000","last_change":"2024-03-07T07:20:08.037898+0000","last_active":"2024-03-07T07:20:09.516748+0000","last_peered":"2024-03-07T07:20:09.516748+0000","last_clean":"2024-03-07T07:20:09.516748+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:09.516748+0000","last_undegraded":"2024-03-07T07:20:09.516748+0000","last_fullsized":"2024-03-07T07:20:09.516748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575696+0000","last_change":"2024-03-07T07:20:08.025458+0000","last_active":"2024-03-07T07:20:09.575696+0000","last_peered":"2024-03-07T07:20:09.575696+0000","last_clean":"2024-03-07T07:20:09.575696+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:09.575696+0000","last_undegraded":"2024-03-07T07:20:09.575696+0000","last_fullsized":"2024-03-07T07:20:09.575696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:24.155245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018155+0000","last_change":"2024-03-07T07:20:08.040577+0000","last_active":"2024-03-07T07:20:09.018155+0000","last_peered":"2024-03-07T07:20:09.018155+0000","last_clean":"2024-03-07T07:20:09.018155+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:09.018155+0000","last_undegraded":"2024-03-07T07:20:09.018155+0000","last_fullsized":"2024-03-07T07:20:09.018155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018038+0000","last_change":"2024-03-07T07:20:08.039803+0000","last_active":"2024-03-07T07:20:09.018038+0000","last_peered":"2024-03-07T07:20:09.018038+0000","last_clean":"2024-03-07T07:20:09.018038+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:09.018038+0000","last_undegraded":"2024-03-07T07:20:09.018038+0000","last_fullsized":"2024-03-07T07:20:09.018038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015233+0000","last_change":"2024-03-07T07:20:08.031321+0000","last_active":"2024-03-07T07:20:09.015233+0000","last_peered":"2024-03-07T07:20:09.015233+0000","last_clean":"2024-03-07T07:20:09.015233+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:09.015233+0000","last_undegraded":"2024-03-07T07:20:09.015233+0000","last_fullsized":"2024-03-07T07:20:09.015233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:51.411320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576999+0000","last_change":"2024-03-07T07:20:08.024172+0000","last_active":"2024-03-07T07:20:09.576999+0000","last_peered":"2024-03-07T07:20:09.576999+0000","last_clean":"2024-03-07T07:20:09.576999+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:09.576999+0000","last_undegraded":"2024-03-07T07:20:09.576999+0000","last_fullsized":"2024-03-07T07:20:09.576999+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576283+0000","last_change":"2024-03-07T07:20:08.033067+0000","last_active":"2024-03-07T07:20:09.576283+0000","last_peered":"2024-03-07T07:20:09.576283+0000","last_clean":"2024-03-07T07:20:09.576283+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:09.576283+0000","last_undegraded":"2024-03-07T07:20:09.576283+0000","last_fullsized":"2024-03-07T07:20:09.576283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.517079+0000","last_change":"2024-03-07T07:20:08.031519+0000","last_active":"2024-03-07T07:20:09.517079+0000","last_peered":"2024-03-07T07:20:09.517079+0000","last_clean":"2024-03-07T07:20:09.517079+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:09.517079+0000","last_undegraded":"2024-03-07T07:20:09.517079+0000","last_fullsized":"2024-03-07T07:20:09.517079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:16:33.954019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576399+0000","last_change":"2024-03-07T07:20:08.043585+0000","last_active":"2024-03-07T07:20:09.576399+0000","last_peered":"2024-03-07T07:20:09.576399+0000","last_clean":"2024-03-07T07:20:09.576399+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:09.576399+0000","last_undegraded":"2024-03-07T07:20:09.576399+0000","last_fullsized":"2024-03-07T07:20:09.576399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576561+0000","last_change":"2024-03-07T07:20:08.021978+0000","last_active":"2024-03-07T07:20:09.576561+0000","last_peered":"2024-03-07T07:20:09.576561+0000","last_clean":"2024-03-07T07:20:09.576561+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:09.576561+0000","last_undegraded":"2024-03-07T07:20:09.576561+0000","last_fullsized":"2024-03-07T07:20:09.576561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018178+0000","last_change":"2024-03-07T07:20:08.040950+0000","last_active":"2024-03-07T07:20:09.018178+0000","last_peered":"2024-03-07T07:20:09.018178+0000","last_clean":"2024-03-07T07:20:09.018178+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:09.018178+0000","last_undegraded":"2024-03-07T07:20:09.018178+0000","last_fullsized":"2024-03-07T07:20:09.018178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576528+0000","last_change":"2024-03-07T07:20:08.031636+0000","last_active":"2024-03-07T07:20:09.576528+0000","last_peered":"2024-03-07T07:20:09.576528+0000","last_clean":"2024-03-07T07:20:09.576528+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:09.576528+0000","last_undegraded":"2024-03-07T07:20:09.576528+0000","last_fullsized":"2024-03-07T07:20:09.576528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017643+0000","last_change":"2024-03-07T07:20:08.023430+0000","last_active":"2024-03-07T07:20:09.017643+0000","last_peered":"2024-03-07T07:20:09.017643+0000","last_clean":"2024-03-07T07:20:09.017643+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:09.017643+0000","last_undegraded":"2024-03-07T07:20:09.017643+0000","last_fullsized":"2024-03-07T07:20:09.017643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:29.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576960+0000","last_change":"2024-03-07T07:20:08.033808+0000","last_active":"2024-03-07T07:20:09.576960+0000","last_peered":"2024-03-07T07:20:09.576960+0000","last_clean":"2024-03-07T07:20:09.576960+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:09.576960+0000","last_undegraded":"2024-03-07T07:20:09.576960+0000","last_fullsized":"2024-03-07T07:20:09.576960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015411+0000","last_change":"2024-03-07T07:20:08.030548+0000","last_active":"2024-03-07T07:20:09.015411+0000","last_peered":"2024-03-07T07:20:09.015411+0000","last_clean":"2024-03-07T07:20:09.015411+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:09.015411+0000","last_undegraded":"2024-03-07T07:20:09.015411+0000","last_fullsized":"2024-03-07T07:20:09.015411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018115+0000","last_change":"2024-03-07T07:20:07.025352+0000","last_active":"2024-03-07T07:20:09.018115+0000","last_peered":"2024-03-07T07:20:09.018115+0000","last_clean":"2024-03-07T07:20:09.018115+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:09.018115+0000","last_undegraded":"2024-03-07T07:20:09.018115+0000","last_fullsized":"2024-03-07T07:20:09.018115+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018004+0000","last_change":"2024-03-07T07:20:08.033169+0000","last_active":"2024-03-07T07:20:09.018004+0000","last_peered":"2024-03-07T07:20:09.018004+0000","last_clean":"2024-03-07T07:20:09.018004+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:09.018004+0000","last_undegraded":"2024-03-07T07:20:09.018004+0000","last_fullsized":"2024-03-07T07:20:09.018004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015349+0000","last_change":"2024-03-07T07:20:07.025175+0000","last_active":"2024-03-07T07:20:09.015349+0000","last_peered":"2024-03-07T07:20:09.015349+0000","last_clean":"2024-03-07T07:20:09.015349+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:09.015349+0000","last_undegraded":"2024-03-07T07:20:09.015349+0000","last_fullsized":"2024-03-07T07:20:09.015349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:09:23.151665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018559+0000","last_change":"2024-03-07T07:20:08.040222+0000","last_active":"2024-03-07T07:20:09.018559+0000","last_peered":"2024-03-07T07:20:09.018559+0000","last_clean":"2024-03-07T07:20:09.018559+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:09.018559+0000","last_undegraded":"2024-03-07T07:20:09.018559+0000","last_fullsized":"2024-03-07T07:20:09.018559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:31.675692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516719+0000","last_change":"2024-03-07T07:20:07.033158+0000","last_active":"2024-03-07T07:20:09.516719+0000","last_peered":"2024-03-07T07:20:09.516719+0000","last_clean":"2024-03-07T07:20:09.516719+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:09.516719+0000","last_undegraded":"2024-03-07T07:20:09.516719+0000","last_fullsized":"2024-03-07T07:20:09.516719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:50:30.672111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576436+0000","last_change":"2024-03-07T07:20:08.043374+0000","last_active":"2024-03-07T07:20:09.576436+0000","last_peered":"2024-03-07T07:20:09.576436+0000","last_clean":"2024-03-07T07:20:09.576436+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:09.576436+0000","last_undegraded":"2024-03-07T07:20:09.576436+0000","last_fullsized":"2024-03-07T07:20:09.576436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575854+0000","last_change":"2024-03-07T07:20:07.028373+0000","last_active":"2024-03-07T07:20:09.575854+0000","last_peered":"2024-03-07T07:20:09.575854+0000","last_clean":"2024-03-07T07:20:09.575854+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:09.575854+0000","last_undegraded":"2024-03-07T07:20:09.575854+0000","last_fullsized":"2024-03-07T07:20:09.575854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:51.229977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576660+0000","last_change":"2024-03-07T07:20:08.033882+0000","last_active":"2024-03-07T07:20:09.576660+0000","last_peered":"2024-03-07T07:20:09.576660+0000","last_clean":"2024-03-07T07:20:09.576660+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:09.576660+0000","last_undegraded":"2024-03-07T07:20:09.576660+0000","last_fullsized":"2024-03-07T07:20:09.576660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:14.532932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018255+0000","last_change":"2024-03-07T07:20:07.028711+0000","last_active":"2024-03-07T07:20:09.018255+0000","last_peered":"2024-03-07T07:20:09.018255+0000","last_clean":"2024-03-07T07:20:09.018255+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:09.018255+0000","last_undegraded":"2024-03-07T07:20:09.018255+0000","last_fullsized":"2024-03-07T07:20:09.018255+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:48:46.879544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576378+0000","last_change":"2024-03-07T07:20:08.030098+0000","last_active":"2024-03-07T07:20:09.576378+0000","last_peered":"2024-03-07T07:20:09.576378+0000","last_clean":"2024-03-07T07:20:09.576378+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:09.576378+0000","last_undegraded":"2024-03-07T07:20:09.576378+0000","last_fullsized":"2024-03-07T07:20:09.576378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018003+0000","last_change":"2024-03-07T07:20:07.029834+0000","last_active":"2024-03-07T07:20:09.018003+0000","last_peered":"2024-03-07T07:20:09.018003+0000","last_clean":"2024-03-07T07:20:09.018003+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:09.018003+0000","last_undegraded":"2024-03-07T07:20:09.018003+0000","last_fullsized":"2024-03-07T07:20:09.018003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:55:49.496286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018248+0000","last_change":"2024-03-07T07:20:08.028053+0000","last_active":"2024-03-07T07:20:09.018248+0000","last_peered":"2024-03-07T07:20:09.018248+0000","last_clean":"2024-03-07T07:20:09.018248+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:09.018248+0000","last_undegraded":"2024-03-07T07:20:09.018248+0000","last_fullsized":"2024-03-07T07:20:09.018248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516477+0000","last_change":"2024-03-07T07:20:07.033411+0000","last_active":"2024-03-07T07:20:09.516477+0000","last_peered":"2024-03-07T07:20:09.516477+0000","last_clean":"2024-03-07T07:20:09.516477+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:09.516477+0000","last_undegraded":"2024-03-07T07:20:09.516477+0000","last_fullsized":"2024-03-07T07:20:09.516477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:34:15.200144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576038+0000","last_change":"2024-03-07T07:20:08.032643+0000","last_active":"2024-03-07T07:20:09.576038+0000","last_peered":"2024-03-07T07:20:09.576038+0000","last_clean":"2024-03-07T07:20:09.576038+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:09.576038+0000","last_undegraded":"2024-03-07T07:20:09.576038+0000","last_fullsized":"2024-03-07T07:20:09.576038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:56:57.505967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018364+0000","last_change":"2024-03-07T07:20:07.027022+0000","last_active":"2024-03-07T07:20:09.018364+0000","last_peered":"2024-03-07T07:20:09.018364+0000","last_clean":"2024-03-07T07:20:09.018364+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:09.018364+0000","last_undegraded":"2024-03-07T07:20:09.018364+0000","last_fullsized":"2024-03-07T07:20:09.018364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:11:35.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.028592+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:14.143287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516961+0000","last_change":"2024-03-07T07:20:07.030367+0000","last_active":"2024-03-07T07:20:09.516961+0000","last_peered":"2024-03-07T07:20:09.516961+0000","last_clean":"2024-03-07T07:20:09.516961+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:09.516961+0000","last_undegraded":"2024-03-07T07:20:09.516961+0000","last_fullsized":"2024-03-07T07:20:09.516961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:12.128114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516860+0000","last_change":"2024-03-07T07:20:08.034949+0000","last_active":"2024-03-07T07:20:09.516860+0000","last_peered":"2024-03-07T07:20:09.516860+0000","last_clean":"2024-03-07T07:20:09.516860+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:09.516860+0000","last_undegraded":"2024-03-07T07:20:09.516860+0000","last_fullsized":"2024-03-07T07:20:09.516860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015409+0000","last_change":"2024-03-07T07:20:07.030049+0000","last_active":"2024-03-07T07:20:09.015409+0000","last_peered":"2024-03-07T07:20:09.015409+0000","last_clean":"2024-03-07T07:20:09.015409+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:09.015409+0000","last_undegraded":"2024-03-07T07:20:09.015409+0000","last_fullsized":"2024-03-07T07:20:09.015409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516872+0000","last_change":"2024-03-07T07:20:08.030884+0000","last_active":"2024-03-07T07:20:09.516872+0000","last_peered":"2024-03-07T07:20:09.516872+0000","last_clean":"2024-03-07T07:20:09.516872+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:09.516872+0000","last_undegraded":"2024-03-07T07:20:09.516872+0000","last_fullsized":"2024-03-07T07:20:09.516872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:22.617306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577049+0000","last_change":"2024-03-07T07:20:07.028526+0000","last_active":"2024-03-07T07:20:09.577049+0000","last_peered":"2024-03-07T07:20:09.577049+0000","last_clean":"2024-03-07T07:20:09.577049+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:09.577049+0000","last_undegraded":"2024-03-07T07:20:09.577049+0000","last_fullsized":"2024-03-07T07:20:09.577049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576521+0000","last_change":"2024-03-07T07:20:08.038594+0000","last_active":"2024-03-07T07:20:09.576521+0000","last_peered":"2024-03-07T07:20:09.576521+0000","last_clean":"2024-03-07T07:20:09.576521+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:09.576521+0000","last_undegraded":"2024-03-07T07:20:09.576521+0000","last_fullsized":"2024-03-07T07:20:09.576521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'10","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:15.856571+0000","last_change":"2024-03-07T07:20:07.035824+0000","last_active":"2024-03-07T07:20:15.856571+0000","last_peered":"2024-03-07T07:20:15.856571+0000","last_clean":"2024-03-07T07:20:15.856571+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:15.856571+0000","last_undegraded":"2024-03-07T07:20:15.856571+0000","last_fullsized":"2024-03-07T07:20:15.856571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7145,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576168+0000","last_change":"2024-03-07T07:20:08.034611+0000","last_active":"2024-03-07T07:20:09.576168+0000","last_peered":"2024-03-07T07:20:09.576168+0000","last_clean":"2024-03-07T07:20:09.576168+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:09.576168+0000","last_undegraded":"2024-03-07T07:20:09.576168+0000","last_fullsized":"2024-03-07T07:20:09.576168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:53:52.459567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018274+0000","last_change":"2024-03-07T07:20:07.025705+0000","last_active":"2024-03-07T07:20:09.018274+0000","last_peered":"2024-03-07T07:20:09.018274+0000","last_clean":"2024-03-07T07:20:09.018274+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:09.018274+0000","last_undegraded":"2024-03-07T07:20:09.018274+0000","last_fullsized":"2024-03-07T07:20:09.018274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516759+0000","last_change":"2024-03-07T07:20:08.037145+0000","last_active":"2024-03-07T07:20:09.516759+0000","last_peered":"2024-03-07T07:20:09.516759+0000","last_clean":"2024-03-07T07:20:09.516759+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:09.516759+0000","last_undegraded":"2024-03-07T07:20:09.516759+0000","last_fullsized":"2024-03-07T07:20:09.516759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:34.249977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576262+0000","last_change":"2024-03-07T07:20:07.026780+0000","last_active":"2024-03-07T07:20:09.576262+0000","last_peered":"2024-03-07T07:20:09.576262+0000","last_clean":"2024-03-07T07:20:09.576262+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:09.576262+0000","last_undegraded":"2024-03-07T07:20:09.576262+0000","last_fullsized":"2024-03-07T07:20:09.576262+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577126+0000","last_change":"2024-03-07T07:20:08.033869+0000","last_active":"2024-03-07T07:20:09.577126+0000","last_peered":"2024-03-07T07:20:09.577126+0000","last_clean":"2024-03-07T07:20:09.577126+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:09.577126+0000","last_undegraded":"2024-03-07T07:20:09.577126+0000","last_fullsized":"2024-03-07T07:20:09.577126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:15:25.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516453+0000","last_change":"2024-03-07T07:20:07.032636+0000","last_active":"2024-03-07T07:20:09.516453+0000","last_peered":"2024-03-07T07:20:09.516453+0000","last_clean":"2024-03-07T07:20:09.516453+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:09.516453+0000","last_undegraded":"2024-03-07T07:20:09.516453+0000","last_fullsized":"2024-03-07T07:20:09.516453+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:51:50.407740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017781+0000","last_change":"2024-03-07T07:20:08.036782+0000","last_active":"2024-03-07T07:20:09.017781+0000","last_peered":"2024-03-07T07:20:09.017781+0000","last_clean":"2024-03-07T07:20:09.017781+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:09.017781+0000","last_undegraded":"2024-03-07T07:20:09.017781+0000","last_fullsized":"2024-03-07T07:20:09.017781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:19.197402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516567+0000","last_change":"2024-03-07T07:20:07.033230+0000","last_active":"2024-03-07T07:20:09.516567+0000","last_peered":"2024-03-07T07:20:09.516567+0000","last_clean":"2024-03-07T07:20:09.516567+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:09.516567+0000","last_undegraded":"2024-03-07T07:20:09.516567+0000","last_fullsized":"2024-03-07T07:20:09.516567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:42.777795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018625+0000","last_change":"2024-03-07T07:20:08.036934+0000","last_active":"2024-03-07T07:20:09.018625+0000","last_peered":"2024-03-07T07:20:09.018625+0000","last_clean":"2024-03-07T07:20:09.018625+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:09.018625+0000","last_undegraded":"2024-03-07T07:20:09.018625+0000","last_fullsized":"2024-03-07T07:20:09.018625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:12.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576460+0000","last_change":"2024-03-07T07:20:07.033018+0000","last_active":"2024-03-07T07:20:09.576460+0000","last_peered":"2024-03-07T07:20:09.576460+0000","last_clean":"2024-03-07T07:20:09.576460+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:09.576460+0000","last_undegraded":"2024-03-07T07:20:09.576460+0000","last_fullsized":"2024-03-07T07:20:09.576460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.577155+0000","last_change":"2024-03-07T07:20:08.035720+0000","last_active":"2024-03-07T07:20:09.577155+0000","last_peered":"2024-03-07T07:20:09.577155+0000","last_clean":"2024-03-07T07:20:09.577155+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:09.577155+0000","last_undegraded":"2024-03-07T07:20:09.577155+0000","last_fullsized":"2024-03-07T07:20:09.577155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:12:58.909222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018650+0000","last_change":"2024-03-07T07:20:07.026560+0000","last_active":"2024-03-07T07:20:09.018650+0000","last_peered":"2024-03-07T07:20:09.018650+0000","last_clean":"2024-03-07T07:20:09.018650+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:09.018650+0000","last_undegraded":"2024-03-07T07:20:09.018650+0000","last_fullsized":"2024-03-07T07:20:09.018650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:30:52.614111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576186+0000","last_change":"2024-03-07T07:20:08.033852+0000","last_active":"2024-03-07T07:20:09.576186+0000","last_peered":"2024-03-07T07:20:09.576186+0000","last_clean":"2024-03-07T07:20:09.576186+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:09.576186+0000","last_undegraded":"2024-03-07T07:20:09.576186+0000","last_fullsized":"2024-03-07T07:20:09.576186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:30.779652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018510+0000","last_change":"2024-03-07T07:20:07.023668+0000","last_active":"2024-03-07T07:20:09.018510+0000","last_peered":"2024-03-07T07:20:09.018510+0000","last_clean":"2024-03-07T07:20:09.018510+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:09.018510+0000","last_undegraded":"2024-03-07T07:20:09.018510+0000","last_fullsized":"2024-03-07T07:20:09.018510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:26:07.665601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575815+0000","last_change":"2024-03-07T07:20:08.030524+0000","last_active":"2024-03-07T07:20:09.575815+0000","last_peered":"2024-03-07T07:20:09.575815+0000","last_clean":"2024-03-07T07:20:09.575815+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:09.575815+0000","last_undegraded":"2024-03-07T07:20:09.575815+0000","last_fullsized":"2024-03-07T07:20:09.575815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:18.342434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018550+0000","last_change":"2024-03-07T07:20:07.029892+0000","last_active":"2024-03-07T07:20:09.018550+0000","last_peered":"2024-03-07T07:20:09.018550+0000","last_clean":"2024-03-07T07:20:09.018550+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:09.018550+0000","last_undegraded":"2024-03-07T07:20:09.018550+0000","last_fullsized":"2024-03-07T07:20:09.018550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:10:24.735113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575612+0000","last_change":"2024-03-07T07:20:08.032134+0000","last_active":"2024-03-07T07:20:09.575612+0000","last_peered":"2024-03-07T07:20:09.575612+0000","last_clean":"2024-03-07T07:20:09.575612+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:09.575612+0000","last_undegraded":"2024-03-07T07:20:09.575612+0000","last_fullsized":"2024-03-07T07:20:09.575612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:48.726782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015375+0000","last_change":"2024-03-07T07:20:07.028153+0000","last_active":"2024-03-07T07:20:09.015375+0000","last_peered":"2024-03-07T07:20:09.015375+0000","last_clean":"2024-03-07T07:20:09.015375+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:09.015375+0000","last_undegraded":"2024-03-07T07:20:09.015375+0000","last_fullsized":"2024-03-07T07:20:09.015375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575806+0000","last_change":"2024-03-07T07:20:08.032594+0000","last_active":"2024-03-07T07:20:09.575806+0000","last_peered":"2024-03-07T07:20:09.575806+0000","last_clean":"2024-03-07T07:20:09.575806+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:09.575806+0000","last_undegraded":"2024-03-07T07:20:09.575806+0000","last_fullsized":"2024-03-07T07:20:09.575806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:15.552375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018303+0000","last_change":"2024-03-07T07:20:07.028455+0000","last_active":"2024-03-07T07:20:09.018303+0000","last_peered":"2024-03-07T07:20:09.018303+0000","last_clean":"2024-03-07T07:20:09.018303+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:09.018303+0000","last_undegraded":"2024-03-07T07:20:09.018303+0000","last_fullsized":"2024-03-07T07:20:09.018303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575541+0000","last_change":"2024-03-07T07:20:08.031299+0000","last_active":"2024-03-07T07:20:09.575541+0000","last_peered":"2024-03-07T07:20:09.575541+0000","last_clean":"2024-03-07T07:20:09.575541+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:09.575541+0000","last_undegraded":"2024-03-07T07:20:09.575541+0000","last_fullsized":"2024-03-07T07:20:09.575541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:24:00.520723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017852+0000","last_change":"2024-03-07T07:20:07.028369+0000","last_active":"2024-03-07T07:20:09.017852+0000","last_peered":"2024-03-07T07:20:09.017852+0000","last_clean":"2024-03-07T07:20:09.017852+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:09.017852+0000","last_undegraded":"2024-03-07T07:20:09.017852+0000","last_fullsized":"2024-03-07T07:20:09.017852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:13:47.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576681+0000","last_change":"2024-03-07T07:20:08.033176+0000","last_active":"2024-03-07T07:20:09.576681+0000","last_peered":"2024-03-07T07:20:09.576681+0000","last_clean":"2024-03-07T07:20:09.576681+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:09.576681+0000","last_undegraded":"2024-03-07T07:20:09.576681+0000","last_fullsized":"2024-03-07T07:20:09.576681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:00.679234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576122+0000","last_change":"2024-03-07T07:20:07.029244+0000","last_active":"2024-03-07T07:20:09.576122+0000","last_peered":"2024-03-07T07:20:09.576122+0000","last_clean":"2024-03-07T07:20:09.576122+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:09.576122+0000","last_undegraded":"2024-03-07T07:20:09.576122+0000","last_fullsized":"2024-03-07T07:20:09.576122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:11.323043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.575628+0000","last_change":"2024-03-07T07:20:08.026414+0000","last_active":"2024-03-07T07:20:09.575628+0000","last_peered":"2024-03-07T07:20:09.575628+0000","last_clean":"2024-03-07T07:20:09.575628+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:09.575628+0000","last_undegraded":"2024-03-07T07:20:09.575628+0000","last_fullsized":"2024-03-07T07:20:09.575628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:51.771656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576502+0000","last_change":"2024-03-07T07:20:07.027164+0000","last_active":"2024-03-07T07:20:09.576502+0000","last_peered":"2024-03-07T07:20:09.576502+0000","last_clean":"2024-03-07T07:20:09.576502+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:09.576502+0000","last_undegraded":"2024-03-07T07:20:09.576502+0000","last_fullsized":"2024-03-07T07:20:09.576502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:01:56.748636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576522+0000","last_change":"2024-03-07T07:20:08.035488+0000","last_active":"2024-03-07T07:20:09.576522+0000","last_peered":"2024-03-07T07:20:09.576522+0000","last_clean":"2024-03-07T07:20:09.576522+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:09.576522+0000","last_undegraded":"2024-03-07T07:20:09.576522+0000","last_fullsized":"2024-03-07T07:20:09.576522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:33.265904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:15.017567+0000","last_change":"2024-03-07T07:20:07.027534+0000","last_active":"2024-03-07T07:20:15.017567+0000","last_peered":"2024-03-07T07:20:15.017567+0000","last_clean":"2024-03-07T07:20:15.017567+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:15.017567+0000","last_undegraded":"2024-03-07T07:20:15.017567+0000","last_fullsized":"2024-03-07T07:20:15.017567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:17.338854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015303+0000","last_change":"2024-03-07T07:20:08.036269+0000","last_active":"2024-03-07T07:20:09.015303+0000","last_peered":"2024-03-07T07:20:09.015303+0000","last_clean":"2024-03-07T07:20:09.015303+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:09.015303+0000","last_undegraded":"2024-03-07T07:20:09.015303+0000","last_fullsized":"2024-03-07T07:20:09.015303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:25:13.131695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576245+0000","last_change":"2024-03-07T07:20:07.023056+0000","last_active":"2024-03-07T07:20:09.576245+0000","last_peered":"2024-03-07T07:20:09.576245+0000","last_clean":"2024-03-07T07:20:09.576245+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:09.576245+0000","last_undegraded":"2024-03-07T07:20:09.576245+0000","last_fullsized":"2024-03-07T07:20:09.576245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:24:14.548795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.018197+0000","last_change":"2024-03-07T07:20:08.023363+0000","last_active":"2024-03-07T07:20:09.018197+0000","last_peered":"2024-03-07T07:20:09.018197+0000","last_clean":"2024-03-07T07:20:09.018197+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:09.018197+0000","last_undegraded":"2024-03-07T07:20:09.018197+0000","last_fullsized":"2024-03-07T07:20:09.018197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:12.326624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015478+0000","last_change":"2024-03-07T07:20:07.024131+0000","last_active":"2024-03-07T07:20:09.015478+0000","last_peered":"2024-03-07T07:20:09.015478+0000","last_clean":"2024-03-07T07:20:09.015478+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:09.015478+0000","last_undegraded":"2024-03-07T07:20:09.015478+0000","last_fullsized":"2024-03-07T07:20:09.015478+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:28.193042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.015390+0000","last_change":"2024-03-07T07:20:08.030007+0000","last_active":"2024-03-07T07:20:09.015390+0000","last_peered":"2024-03-07T07:20:09.015390+0000","last_clean":"2024-03-07T07:20:09.015390+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:09.015390+0000","last_undegraded":"2024-03-07T07:20:09.015390+0000","last_fullsized":"2024-03-07T07:20:09.015390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:43.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576311+0000","last_change":"2024-03-07T07:20:07.025544+0000","last_active":"2024-03-07T07:20:09.576311+0000","last_peered":"2024-03-07T07:20:09.576311+0000","last_clean":"2024-03-07T07:20:09.576311+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:09.576311+0000","last_undegraded":"2024-03-07T07:20:09.576311+0000","last_fullsized":"2024-03-07T07:20:09.576311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:04:18.193822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516464+0000","last_change":"2024-03-07T07:20:08.035274+0000","last_active":"2024-03-07T07:20:09.516464+0000","last_peered":"2024-03-07T07:20:09.516464+0000","last_clean":"2024-03-07T07:20:09.516464+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:09.516464+0000","last_undegraded":"2024-03-07T07:20:09.516464+0000","last_fullsized":"2024-03-07T07:20:09.516464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:10:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017946+0000","last_change":"2024-03-07T07:20:07.023385+0000","last_active":"2024-03-07T07:20:09.017946+0000","last_peered":"2024-03-07T07:20:09.017946+0000","last_clean":"2024-03-07T07:20:09.017946+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:09.017946+0000","last_undegraded":"2024-03-07T07:20:09.017946+0000","last_fullsized":"2024-03-07T07:20:09.017946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:09:50.768076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017838+0000","last_change":"2024-03-07T07:20:08.040036+0000","last_active":"2024-03-07T07:20:09.017838+0000","last_peered":"2024-03-07T07:20:09.017838+0000","last_clean":"2024-03-07T07:20:09.017838+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:09.017838+0000","last_undegraded":"2024-03-07T07:20:09.017838+0000","last_fullsized":"2024-03-07T07:20:09.017838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:21.299779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516475+0000","last_change":"2024-03-07T07:20:07.036824+0000","last_active":"2024-03-07T07:20:09.516475+0000","last_peered":"2024-03-07T07:20:09.516475+0000","last_clean":"2024-03-07T07:20:09.516475+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:09.516475+0000","last_undegraded":"2024-03-07T07:20:09.516475+0000","last_fullsized":"2024-03-07T07:20:09.516475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:51:20.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576659+0000","last_change":"2024-03-07T07:20:08.034942+0000","last_active":"2024-03-07T07:20:09.576659+0000","last_peered":"2024-03-07T07:20:09.576659+0000","last_clean":"2024-03-07T07:20:09.576659+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:09.576659+0000","last_undegraded":"2024-03-07T07:20:09.576659+0000","last_fullsized":"2024-03-07T07:20:09.576659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:59:54.123007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.017654+0000","last_change":"2024-03-07T07:20:07.023606+0000","last_active":"2024-03-07T07:20:09.017654+0000","last_peered":"2024-03-07T07:20:09.017654+0000","last_clean":"2024-03-07T07:20:09.017654+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:09.017654+0000","last_undegraded":"2024-03-07T07:20:09.017654+0000","last_fullsized":"2024-03-07T07:20:09.017654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:06.002141+0000","last_clean_scrub_stamp":"2024-03-07T07:20:06.002141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:23:59.517142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.576204+0000","last_change":"2024-03-07T07:19:05.376936+0000","last_active":"2024-03-07T07:20:09.576204+0000","last_peered":"2024-03-07T07:20:09.576204+0000","last_clean":"2024-03-07T07:20:09.576204+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:09.576204+0000","last_undegraded":"2024-03-07T07:20:09.576204+0000","last_fullsized":"2024-03-07T07:20:09.576204+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:19:04.247114+0000","last_clean_scrub_stamp":"2024-03-07T07:19:04.247114+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:00:09.303910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":7943,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":15886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:20:25.679 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.680 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.680 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.680 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.680 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.680 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.680 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.681 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.681 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.681 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.681 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.681 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.681 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.682 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.682 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.682 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.682 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.682 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.682 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.683 INFO:tasks.ceph:pgid 5.2c last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.683 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.683 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.683 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.683 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.683 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.684 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.684 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.684 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.684 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.684 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.684 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.685 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.685 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.685 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.685 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.685 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.685 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.685 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.686 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.686 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.686 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.686 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.686 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.686 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.687 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.687 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.687 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.687 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.687 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.687 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.688 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.688 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.688 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.688 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.688 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.689 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.689 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.689 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.689 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.689 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.689 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.689 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.690 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.690 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.690 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.690 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.690 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.690 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.691 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.691 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.691 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.691 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.691 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.691 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.692 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.692 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.692 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.692 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.692 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.692 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.693 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.693 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.693 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.693 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.693 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.693 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.694 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.694 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.694 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.694 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.694 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.694 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.695 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.695 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.695 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.695 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-03-07T07:20:06.002141+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.695 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-07T07:19:04.247114+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=19, tm_sec=4, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:25.695 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-07T07:20:45.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:20:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.799+0000 7fd1b4634640 1 Processor -- start 2024-03-07T07:20:45.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.799+0000 7fd1b4634640 1 -- start start 2024-03-07T07:20:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b4634640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd1ac144540 0x7fd1ac162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b4634640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1ac162ab0 0x7fd1ac071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b4634640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0b4bd0 con 0x7fd1ac0b5dd0 2024-03-07T07:20:45.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b4634640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac141700 con 0x7fd1ac162ab0 2024-03-07T07:20:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b4634640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac161e00 con 0x7fd1ac144540 2024-03-07T07:20:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b2bd1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1ac162ab0 0x7fd1ac071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:45.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b2bd1640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7fd1ac162ab0 0x7fd1ac071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39174/0 (socket says 172.21.15.64:39174) 2024-03-07T07:20:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b2bd1640 1 -- 172.21.15.64:0/2897597951 learned_addr learned my addr 172.21.15.64:0/2897597951 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b1bcf640 1 --2- 172.21.15.64:0/2897597951 >> v2:172.21.15.120:3300/0 conn(0x7fd1ac144540 0x7fd1ac162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b2bd1640 1 -- 172.21.15.64:0/2897597951 >> v2:172.21.15.120:3300/0 conn(0x7fd1ac144540 msgr2=0x7fd1ac162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:20:45.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b2bd1640 1 --2- 172.21.15.64:0/2897597951 >> v2:172.21.15.120:3300/0 conn(0x7fd1ac144540 0x7fd1ac162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b2bd1640 1 -- 172.21.15.64:0/2897597951 >> v1:172.21.15.120:6789/0 conn(0x7fd1ac0b5dd0 legacy=0x7fd1ac0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b2bd1640 1 -- 172.21.15.64:0/2897597951 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0b4eb0 con 0x7fd1ac162ab0 2024-03-07T07:20:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.800+0000 7fd1b1bcf640 1 --2- 172.21.15.64:0/2897597951 >> v2:172.21.15.120:3300/0 conn(0x7fd1ac144540 0x7fd1ac162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:45.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.801+0000 7fd1b2bd1640 1 --2- 172.21.15.64:0/2897597951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac162ab0 0x7fd1ac071410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac08d440 tx=0x7fd1a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9a14e7dd40984e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:45.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.801+0000 7fd1b13ce640 1 -- 172.21.15.64:0/2897597951 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a402ca80 con 0x7fd1ac162ab0 2024-03-07T07:20:45.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.801+0000 7fd1b13ce640 1 -- 172.21.15.64:0/2897597951 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a402cc20 con 0x7fd1ac162ab0 2024-03-07T07:20:45.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.801+0000 7fd1b13ce640 1 -- 172.21.15.64:0/2897597951 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a40343f0 con 0x7fd1ac162ab0 2024-03-07T07:20:45.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 -- 172.21.15.64:0/2897597951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac162ab0 msgr2=0x7fd1ac071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:45.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 --2- 172.21.15.64:0/2897597951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac162ab0 0x7fd1ac071410 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac08d440 tx=0x7fd1a402bee0 comp rx=0 tx=0).stop 2024-03-07T07:20:45.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 -- 172.21.15.64:0/2897597951 shutdown_connections 2024-03-07T07:20:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 --2- 172.21.15.64:0/2897597951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac162ab0 0x7fd1ac071410 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 --2- 172.21.15.64:0/2897597951 >> v2:172.21.15.120:3300/0 conn(0x7fd1ac144540 0x7fd1ac162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 -- 172.21.15.64:0/2897597951 >> 172.21.15.64:0/2897597951 conn(0x7fd1ac0d0e70 msgr2=0x7fd1ac0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:45.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 -- 172.21.15.64:0/2897597951 shutdown_connections 2024-03-07T07:20:45.809 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.802+0000 7fd1b4634640 1 -- 172.21.15.64:0/2897597951 wait complete. 2024-03-07T07:20:45.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.803+0000 7fd1b4634640 1 Processor -- start 2024-03-07T07:20:45.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.803+0000 7fd1b4634640 1 -- start start 2024-03-07T07:20:45.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b4634640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1ac0b5dd0 0x7fd1ac15b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b4634640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac144540 0x7fd1ac159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b4634640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1f4f00 0x7fd1ac13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b4634640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0b61b0 con 0x7fd1ac144540 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b4634640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac162050 con 0x7fd1ac1f4f00 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b4634640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac071950 con 0x7fd1ac0b5dd0 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac144540 0x7fd1ac159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac144540 0x7fd1ac159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39182/0 (socket says 172.21.15.64:39182) 2024-03-07T07:20:45.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 -- 172.21.15.64:0/1623629530 learned_addr learned my addr 172.21.15.64:0/1623629530 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b2bd1640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1f4f00 0x7fd1ac13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b23d0640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1ac0b5dd0 0x7fd1ac15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 -- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1ac0b5dd0 msgr2=0x7fd1ac15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1ac0b5dd0 0x7fd1ac15b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 -- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1f4f00 msgr2=0x7fd1ac13e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1f4f00 0x7fd1ac13e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 -- 172.21.15.64:0/1623629530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac13d750 con 0x7fd1ac144540 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b23d0640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1ac0b5dd0 0x7fd1ac15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:20:45.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b1bcf640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac144540 0x7fd1ac159a30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00e7e0 tx=0x7fd19c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1a2ffd640 1 -- 172.21.15.64:0/1623629530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00f040 con 0x7fd1ac144540 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1a2ffd640 1 -- 172.21.15.64:0/1623629530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c004030 con 0x7fd1ac144540 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac13da30 con 0x7fd1ac144540 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.804+0000 7fd1a2ffd640 1 -- 172.21.15.64:0/1623629530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c004400 con 0x7fd1ac144540 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.805+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac13e2d0 con 0x7fd1ac144540 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.805+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac13dc00 con 0x7fd1ac144540 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.806+0000 7fd1a2ffd640 1 -- 172.21.15.64:0/1623629530 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd19c00c6d0 con 0x7fd1ac144540 2024-03-07T07:20:45.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.806+0000 7fd1a2ffd640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd19006b500 0x7fd19006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:20:45.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.806+0000 7fd1a2ffd640 1 -- 172.21.15.64:0/1623629530 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd19c089f20 con 0x7fd1ac144540 2024-03-07T07:20:45.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.806+0000 7fd1b23d0640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd19006b500 0x7fd19006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:20:45.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.807+0000 7fd1b23d0640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd19006b500 0x7fd19006d9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8001700 tx=0x7fd1a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:20:45.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.810+0000 7fd1a2ffd640 1 -- 172.21.15.64:0/1623629530 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd19c017020 con 0x7fd1ac144540 2024-03-07T07:20:45.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.988+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1ac062c90 con 0x7fd19006b500 2024-03-07T07:20:45.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.995+0000 7fd1a2ffd640 1 -- 172.21.15.64:0/1623629530 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251741 (secure 0 0 0) 0x7fd1ac062c90 con 0x7fd19006b500 2024-03-07T07:20:45.997 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:20:45.999 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:20:46.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd19006b500 msgr2=0x7fd19006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:46.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd19006b500 0x7fd19006d9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8001700 tx=0x7fd1a8009040 comp rx=0 tx=0).stop 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac144540 msgr2=0x7fd1ac159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac144540 0x7fd1ac159a30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00e7e0 tx=0x7fd19c00eca0 comp rx=0 tx=0).stop 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 shutdown_connections 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7fd19006b500 0x7fd19006d9c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1ac144540 0x7fd1ac159a30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1ac1f4f00 0x7fd1ac13e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 --2- 172.21.15.64:0/1623629530 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1ac0b5dd0 0x7fd1ac15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:20:46.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 >> 172.21.15.64:0/1623629530 conn(0x7fd1ac0d0e70 msgr2=0x7fd1ac15f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:20:46.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 shutdown_connections 2024-03-07T07:20:46.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:20:45.998+0000 7fd1b4634640 1 -- 172.21.15.64:0/1623629530 wait complete. 2024-03-07T07:20:46.009 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-07T07:20:44.198247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598311,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220272,"kb_used_data":4144,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754448,"statfs":{"total":773094113280,"available":772868554752,"internally_reserved":0,"allocated":4243456,"data_stored":2369550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003050"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.270024+0000","last_change":"2024-03-07T07:20:30.270046+0000","last_active":"2024-03-07T07:20:30.270024+0000","last_peered":"2024-03-07T07:20:30.270024+0000","last_clean":"2024-03-07T07:20:30.270024+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:30.270024+0000","last_undegraded":"2024-03-07T07:20:30.270024+0000","last_fullsized":"2024-03-07T07:20:30.270024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.269932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.269932+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.269932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:45.881517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:38.302425+0000","last_change":"2024-03-07T07:20:38.302447+0000","last_active":"2024-03-07T07:20:38.302425+0000","last_peered":"2024-03-07T07:20:38.302425+0000","last_clean":"2024-03-07T07:20:38.302425+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:38.302425+0000","last_undegraded":"2024-03-07T07:20:38.302425+0000","last_fullsized":"2024-03-07T07:20:38.302425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:38.302352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:38.302352+0000","last_clean_scrub_stamp":"2024-03-07T07:20:38.302352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:32:29.833779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.821680+0000","last_change":"2024-03-07T07:20:34.821726+0000","last_active":"2024-03-07T07:20:34.821680+0000","last_peered":"2024-03-07T07:20:34.821680+0000","last_clean":"2024-03-07T07:20:34.821680+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:34.821680+0000","last_undegraded":"2024-03-07T07:20:34.821680+0000","last_fullsized":"2024-03-07T07:20:34.821680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.821575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.821575+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.821575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:28:27.159052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.507728+0000","last_change":"2024-03-07T07:20:32.507769+0000","last_active":"2024-03-07T07:20:32.507728+0000","last_peered":"2024-03-07T07:20:32.507728+0000","last_clean":"2024-03-07T07:20:32.507728+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:32.507728+0000","last_undegraded":"2024-03-07T07:20:32.507728+0000","last_fullsized":"2024-03-07T07:20:32.507728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.507639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.507639+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.507639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:28.788762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.361831+0000","last_change":"2024-03-07T07:20:31.361869+0000","last_active":"2024-03-07T07:20:31.361831+0000","last_peered":"2024-03-07T07:20:31.361831+0000","last_clean":"2024-03-07T07:20:31.361831+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:31.361831+0000","last_undegraded":"2024-03-07T07:20:31.361831+0000","last_fullsized":"2024-03-07T07:20:31.361831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.361723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.361723+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.361723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:38.499288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.503767+0000","last_change":"2024-03-07T07:20:24.503806+0000","last_active":"2024-03-07T07:20:24.503767+0000","last_peered":"2024-03-07T07:20:24.503767+0000","last_clean":"2024-03-07T07:20:24.503767+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:24.503767+0000","last_undegraded":"2024-03-07T07:20:24.503767+0000","last_fullsized":"2024-03-07T07:20:24.503767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.503653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.503653+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.503653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:46:08.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.460317+0000","last_change":"2024-03-07T07:20:27.460341+0000","last_active":"2024-03-07T07:20:27.460317+0000","last_peered":"2024-03-07T07:20:27.460317+0000","last_clean":"2024-03-07T07:20:27.460317+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:27.460317+0000","last_undegraded":"2024-03-07T07:20:27.460317+0000","last_fullsized":"2024-03-07T07:20:27.460317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.460227+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:09:10.521273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.412734+0000","last_change":"2024-03-07T07:20:34.412773+0000","last_active":"2024-03-07T07:20:34.412734+0000","last_peered":"2024-03-07T07:20:34.412734+0000","last_clean":"2024-03-07T07:20:34.412734+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:34.412734+0000","last_undegraded":"2024-03-07T07:20:34.412734+0000","last_fullsized":"2024-03-07T07:20:34.412734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.412603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.412603+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.412603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:20.470367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.235483+0000","last_change":"2024-03-07T07:20:34.235509+0000","last_active":"2024-03-07T07:20:34.235483+0000","last_peered":"2024-03-07T07:20:34.235483+0000","last_clean":"2024-03-07T07:20:34.235483+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:34.235483+0000","last_undegraded":"2024-03-07T07:20:34.235483+0000","last_fullsized":"2024-03-07T07:20:34.235483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.235381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.235381+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.235381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:03:19.293627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.533766+0000","last_change":"2024-03-07T07:20:31.533793+0000","last_active":"2024-03-07T07:20:31.533766+0000","last_peered":"2024-03-07T07:20:31.533766+0000","last_clean":"2024-03-07T07:20:31.533766+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:31.533766+0000","last_undegraded":"2024-03-07T07:20:31.533766+0000","last_fullsized":"2024-03-07T07:20:31.533766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.533698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.533698+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.533698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:44.193713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:36.475255+0000","last_change":"2024-03-07T07:20:36.475278+0000","last_active":"2024-03-07T07:20:36.475255+0000","last_peered":"2024-03-07T07:20:36.475255+0000","last_clean":"2024-03-07T07:20:36.475255+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:36.475255+0000","last_undegraded":"2024-03-07T07:20:36.475255+0000","last_fullsized":"2024-03-07T07:20:36.475255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:36.475163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:36.475163+0000","last_clean_scrub_stamp":"2024-03-07T07:20:36.475163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:57:59.146903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.974491+0000","last_change":"2024-03-07T07:20:29.974546+0000","last_active":"2024-03-07T07:20:29.974491+0000","last_peered":"2024-03-07T07:20:29.974491+0000","last_clean":"2024-03-07T07:20:29.974491+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:29.974491+0000","last_undegraded":"2024-03-07T07:20:29.974491+0000","last_fullsized":"2024-03-07T07:20:29.974491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.974395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.974395+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.974395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:07:35.107200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.363449+0000","last_change":"2024-03-07T07:20:30.363468+0000","last_active":"2024-03-07T07:20:30.363449+0000","last_peered":"2024-03-07T07:20:30.363449+0000","last_clean":"2024-03-07T07:20:30.363449+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:30.363449+0000","last_undegraded":"2024-03-07T07:20:30.363449+0000","last_fullsized":"2024-03-07T07:20:30.363449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.363368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.363368+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.363368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:11:04.092642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.408466+0000","last_change":"2024-03-07T07:20:32.408489+0000","last_active":"2024-03-07T07:20:32.408466+0000","last_peered":"2024-03-07T07:20:32.408466+0000","last_clean":"2024-03-07T07:20:32.408466+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:32.408466+0000","last_undegraded":"2024-03-07T07:20:32.408466+0000","last_fullsized":"2024-03-07T07:20:32.408466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.408389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.408389+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.408389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:18.466153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.048120+0000","last_change":"2024-03-07T07:20:27.048143+0000","last_active":"2024-03-07T07:20:27.048120+0000","last_peered":"2024-03-07T07:20:27.048120+0000","last_clean":"2024-03-07T07:20:27.048120+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:27.048120+0000","last_undegraded":"2024-03-07T07:20:27.048120+0000","last_fullsized":"2024-03-07T07:20:27.048120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.047987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.047987+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.047987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:09:10.109033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.438565+0000","last_change":"2024-03-07T07:20:35.438587+0000","last_active":"2024-03-07T07:20:35.438565+0000","last_peered":"2024-03-07T07:20:35.438565+0000","last_clean":"2024-03-07T07:20:35.438565+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:35.438565+0000","last_undegraded":"2024-03-07T07:20:35.438565+0000","last_fullsized":"2024-03-07T07:20:35.438565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.438466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.438466+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.438466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:28:27.775943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.435073+0000","last_change":"2024-03-07T07:20:32.435100+0000","last_active":"2024-03-07T07:20:32.435073+0000","last_peered":"2024-03-07T07:20:32.435073+0000","last_clean":"2024-03-07T07:20:32.435073+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:32.435073+0000","last_undegraded":"2024-03-07T07:20:32.435073+0000","last_fullsized":"2024-03-07T07:20:32.435073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.434962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.434962+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.434962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:44:38.521587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:37.349723+0000","last_change":"2024-03-07T07:20:37.349750+0000","last_active":"2024-03-07T07:20:37.349723+0000","last_peered":"2024-03-07T07:20:37.349723+0000","last_clean":"2024-03-07T07:20:37.349723+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:37.349723+0000","last_undegraded":"2024-03-07T07:20:37.349723+0000","last_fullsized":"2024-03-07T07:20:37.349723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:37.349647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:37.349647+0000","last_clean_scrub_stamp":"2024-03-07T07:20:37.349647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:20:08.492778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516934+0000","last_change":"2024-03-07T07:20:08.037898+0000","last_active":"2024-03-07T07:20:09.516934+0000","last_peered":"2024-03-07T07:20:09.516934+0000","last_clean":"2024-03-07T07:20:09.516934+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:09.516934+0000","last_undegraded":"2024-03-07T07:20:09.516934+0000","last_fullsized":"2024-03-07T07:20:09.516934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.446127+0000","last_change":"2024-03-07T07:20:24.446147+0000","last_active":"2024-03-07T07:20:24.446127+0000","last_peered":"2024-03-07T07:20:24.446127+0000","last_clean":"2024-03-07T07:20:24.446127+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:24.446127+0000","last_undegraded":"2024-03-07T07:20:24.446127+0000","last_fullsized":"2024-03-07T07:20:24.446127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.446044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.446044+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.446044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:39:01.221698+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.199639+0000","last_change":"2024-03-07T07:20:08.040577+0000","last_active":"2024-03-07T07:20:09.199639+0000","last_peered":"2024-03-07T07:20:09.199639+0000","last_clean":"2024-03-07T07:20:09.199639+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:09.199639+0000","last_undegraded":"2024-03-07T07:20:09.199639+0000","last_fullsized":"2024-03-07T07:20:09.199639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:36.147437+0000","last_change":"2024-03-07T07:20:36.147464+0000","last_active":"2024-03-07T07:20:36.147437+0000","last_peered":"2024-03-07T07:20:36.147437+0000","last_clean":"2024-03-07T07:20:36.147437+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:36.147437+0000","last_undegraded":"2024-03-07T07:20:36.147437+0000","last_fullsized":"2024-03-07T07:20:36.147437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:36.147347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:36.147347+0000","last_clean_scrub_stamp":"2024-03-07T07:20:36.147347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:07:41.280152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.551867+0000","last_change":"2024-03-07T07:20:29.551890+0000","last_active":"2024-03-07T07:20:29.551867+0000","last_peered":"2024-03-07T07:20:29.551867+0000","last_clean":"2024-03-07T07:20:29.551867+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:29.551867+0000","last_undegraded":"2024-03-07T07:20:29.551867+0000","last_fullsized":"2024-03-07T07:20:29.551867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.551797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.551797+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.551797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:56.336512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.003735+0000","last_change":"2024-03-07T07:20:29.003759+0000","last_active":"2024-03-07T07:20:29.003735+0000","last_peered":"2024-03-07T07:20:29.003735+0000","last_clean":"2024-03-07T07:20:29.003735+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:29.003735+0000","last_undegraded":"2024-03-07T07:20:29.003735+0000","last_fullsized":"2024-03-07T07:20:29.003735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.003642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.003642+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.003642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:31:02.519136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.497765+0000","last_change":"2024-03-07T07:20:28.497801+0000","last_active":"2024-03-07T07:20:28.497765+0000","last_peered":"2024-03-07T07:20:28.497765+0000","last_clean":"2024-03-07T07:20:28.497765+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:28.497765+0000","last_undegraded":"2024-03-07T07:20:28.497765+0000","last_fullsized":"2024-03-07T07:20:28.497765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.497661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.497661+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.497661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:16.831116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.416477+0000","last_change":"2024-03-07T07:20:34.416506+0000","last_active":"2024-03-07T07:20:34.416477+0000","last_peered":"2024-03-07T07:20:34.416477+0000","last_clean":"2024-03-07T07:20:34.416477+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:34.416477+0000","last_undegraded":"2024-03-07T07:20:34.416477+0000","last_fullsized":"2024-03-07T07:20:34.416477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.416399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.416399+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.416399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:13:02.792116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.493156+0000","last_change":"2024-03-07T07:20:34.493193+0000","last_active":"2024-03-07T07:20:34.493156+0000","last_peered":"2024-03-07T07:20:34.493156+0000","last_clean":"2024-03-07T07:20:34.493156+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:34.493156+0000","last_undegraded":"2024-03-07T07:20:34.493156+0000","last_fullsized":"2024-03-07T07:20:34.493156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.493061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.493061+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.493061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:20.550825+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.035852+0000","last_change":"2024-03-07T07:20:28.035891+0000","last_active":"2024-03-07T07:20:28.035852+0000","last_peered":"2024-03-07T07:20:28.035852+0000","last_clean":"2024-03-07T07:20:28.035852+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:28.035852+0000","last_undegraded":"2024-03-07T07:20:28.035852+0000","last_fullsized":"2024-03-07T07:20:28.035852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.035749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.035749+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.035749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:16.369204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.199646+0000","last_change":"2024-03-07T07:20:08.040950+0000","last_active":"2024-03-07T07:20:09.199646+0000","last_peered":"2024-03-07T07:20:09.199646+0000","last_clean":"2024-03-07T07:20:09.199646+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:09.199646+0000","last_undegraded":"2024-03-07T07:20:09.199646+0000","last_fullsized":"2024-03-07T07:20:09.199646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.458572+0000","last_change":"2024-03-07T07:20:26.458614+0000","last_active":"2024-03-07T07:20:26.458572+0000","last_peered":"2024-03-07T07:20:26.458572+0000","last_clean":"2024-03-07T07:20:26.458572+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:26.458572+0000","last_undegraded":"2024-03-07T07:20:26.458572+0000","last_fullsized":"2024-03-07T07:20:26.458572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.458474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.458474+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.458474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:03.393547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.254635+0000","last_change":"2024-03-07T07:20:29.254656+0000","last_active":"2024-03-07T07:20:29.254635+0000","last_peered":"2024-03-07T07:20:29.254635+0000","last_clean":"2024-03-07T07:20:29.254635+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:29.254635+0000","last_undegraded":"2024-03-07T07:20:29.254635+0000","last_fullsized":"2024-03-07T07:20:29.254635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.254540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.254540+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.254540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:56.498796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.493976+0000","last_change":"2024-03-07T07:20:29.493999+0000","last_active":"2024-03-07T07:20:29.493976+0000","last_peered":"2024-03-07T07:20:29.493976+0000","last_clean":"2024-03-07T07:20:29.493976+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:29.493976+0000","last_undegraded":"2024-03-07T07:20:29.493976+0000","last_fullsized":"2024-03-07T07:20:29.493976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.493902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.493902+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.493902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:31:03.009396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.538255+0000","last_change":"2024-03-07T07:20:28.538294+0000","last_active":"2024-03-07T07:20:28.538255+0000","last_peered":"2024-03-07T07:20:28.538255+0000","last_clean":"2024-03-07T07:20:28.538255+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:28.538255+0000","last_undegraded":"2024-03-07T07:20:28.538255+0000","last_fullsized":"2024-03-07T07:20:28.538255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.538127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.538127+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.538127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:04.576347+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.260297+0000","last_change":"2024-03-07T07:20:26.260329+0000","last_active":"2024-03-07T07:20:26.260297+0000","last_peered":"2024-03-07T07:20:26.260297+0000","last_clean":"2024-03-07T07:20:26.260297+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:26.260297+0000","last_undegraded":"2024-03-07T07:20:26.260297+0000","last_fullsized":"2024-03-07T07:20:26.260297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-03-07T07:20:26.260206+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-03-07T07:20:26.260206+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.260206+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:50.034136+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.275374+0000","last_change":"2024-03-07T07:20:31.275409+0000","last_active":"2024-03-07T07:20:31.275374+0000","last_peered":"2024-03-07T07:20:31.275374+0000","last_clean":"2024-03-07T07:20:31.275374+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:31.275374+0000","last_undegraded":"2024-03-07T07:20:31.275374+0000","last_fullsized":"2024-03-07T07:20:31.275374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.275252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.275252+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.275252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:31.725386+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:23.594241+0000","last_change":"2024-03-07T07:20:23.594281+0000","last_active":"2024-03-07T07:20:23.594241+0000","last_peered":"2024-03-07T07:20:23.594241+0000","last_clean":"2024-03-07T07:20:23.594241+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:23.594241+0000","last_undegraded":"2024-03-07T07:20:23.594241+0000","last_fullsized":"2024-03-07T07:20:23.594241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:23.594133+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:23.594133+0000","last_clean_scrub_stamp":"2024-03-07T07:20:23.594133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:31.121344+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:38.219809+0000","last_change":"2024-03-07T07:20:38.219832+0000","last_active":"2024-03-07T07:20:38.219809+0000","last_peered":"2024-03-07T07:20:38.219809+0000","last_clean":"2024-03-07T07:20:38.219809+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:38.219809+0000","last_undegraded":"2024-03-07T07:20:38.219809+0000","last_fullsized":"2024-03-07T07:20:38.219809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:38.219718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:38.219718+0000","last_clean_scrub_stamp":"2024-03-07T07:20:38.219718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:44:44.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.468303+0000","last_change":"2024-03-07T07:20:27.468352+0000","last_active":"2024-03-07T07:20:27.468303+0000","last_peered":"2024-03-07T07:20:27.468303+0000","last_clean":"2024-03-07T07:20:27.468303+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:27.468303+0000","last_undegraded":"2024-03-07T07:20:27.468303+0000","last_fullsized":"2024-03-07T07:20:27.468303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:27.468188+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:27.468188+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.468188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:06.301869+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.492596+0000","last_change":"2024-03-07T07:20:33.492650+0000","last_active":"2024-03-07T07:20:33.492596+0000","last_peered":"2024-03-07T07:20:33.492596+0000","last_clean":"2024-03-07T07:20:33.492596+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:33.492596+0000","last_undegraded":"2024-03-07T07:20:33.492596+0000","last_fullsized":"2024-03-07T07:20:33.492596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.492487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.492487+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.492487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:13.308386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:23.470674+0000","last_change":"2024-03-07T07:20:23.470722+0000","last_active":"2024-03-07T07:20:23.470674+0000","last_peered":"2024-03-07T07:20:23.470674+0000","last_clean":"2024-03-07T07:20:23.470674+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:23.470674+0000","last_undegraded":"2024-03-07T07:20:23.470674+0000","last_fullsized":"2024-03-07T07:20:23.470674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:23.470554+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:23.470554+0000","last_clean_scrub_stamp":"2024-03-07T07:20:23.470554+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:52:07.876153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.439009+0000","last_change":"2024-03-07T07:20:31.439043+0000","last_active":"2024-03-07T07:20:31.439009+0000","last_peered":"2024-03-07T07:20:31.439009+0000","last_clean":"2024-03-07T07:20:31.439009+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:31.439009+0000","last_undegraded":"2024-03-07T07:20:31.439009+0000","last_fullsized":"2024-03-07T07:20:31.439009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.438907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.438907+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.438907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:34.046498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.248261+0000","last_change":"2024-03-07T07:20:31.248309+0000","last_active":"2024-03-07T07:20:31.248261+0000","last_peered":"2024-03-07T07:20:31.248261+0000","last_clean":"2024-03-07T07:20:31.248261+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:31.248261+0000","last_undegraded":"2024-03-07T07:20:31.248261+0000","last_fullsized":"2024-03-07T07:20:31.248261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:31.248136+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:31.248136+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.248136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:10.388114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.921627+0000","last_change":"2024-03-07T07:20:31.921665+0000","last_active":"2024-03-07T07:20:31.921627+0000","last_peered":"2024-03-07T07:20:31.921627+0000","last_clean":"2024-03-07T07:20:31.921627+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:31.921627+0000","last_undegraded":"2024-03-07T07:20:31.921627+0000","last_fullsized":"2024-03-07T07:20:31.921627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.921523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.921523+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.921523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:28.202647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.279481+0000","last_change":"2024-03-07T07:20:32.279506+0000","last_active":"2024-03-07T07:20:32.279481+0000","last_peered":"2024-03-07T07:20:32.279481+0000","last_clean":"2024-03-07T07:20:32.279481+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:32.279481+0000","last_undegraded":"2024-03-07T07:20:32.279481+0000","last_fullsized":"2024-03-07T07:20:32.279481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:32.279387+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:32.279387+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.279387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:09.214460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.280869+0000","last_change":"2024-03-07T07:20:33.280913+0000","last_active":"2024-03-07T07:20:33.280869+0000","last_peered":"2024-03-07T07:20:33.280869+0000","last_clean":"2024-03-07T07:20:33.280869+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:33.280869+0000","last_undegraded":"2024-03-07T07:20:33.280869+0000","last_fullsized":"2024-03-07T07:20:33.280869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.280770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.280770+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.280770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:09:16.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.469330+0000","last_change":"2024-03-07T07:20:29.469350+0000","last_active":"2024-03-07T07:20:29.469330+0000","last_peered":"2024-03-07T07:20:29.469330+0000","last_clean":"2024-03-07T07:20:29.469330+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:29.469330+0000","last_undegraded":"2024-03-07T07:20:29.469330+0000","last_fullsized":"2024-03-07T07:20:29.469330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.469266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.469266+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.469266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:42.129281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.514425+0000","last_change":"2024-03-07T07:20:27.514461+0000","last_active":"2024-03-07T07:20:27.514425+0000","last_peered":"2024-03-07T07:20:27.514425+0000","last_clean":"2024-03-07T07:20:27.514425+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:27.514425+0000","last_undegraded":"2024-03-07T07:20:27.514425+0000","last_fullsized":"2024-03-07T07:20:27.514425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.514313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.514313+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.514313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:04:48.750866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.329249+0000","last_change":"2024-03-07T07:20:29.329276+0000","last_active":"2024-03-07T07:20:29.329249+0000","last_peered":"2024-03-07T07:20:29.329249+0000","last_clean":"2024-03-07T07:20:29.329249+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:29.329249+0000","last_undegraded":"2024-03-07T07:20:29.329249+0000","last_fullsized":"2024-03-07T07:20:29.329249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.329164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.329164+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.329164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:29.779298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.958179+0000","last_change":"2024-03-07T07:20:30.958214+0000","last_active":"2024-03-07T07:20:30.958179+0000","last_peered":"2024-03-07T07:20:30.958179+0000","last_clean":"2024-03-07T07:20:30.958179+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:30.958179+0000","last_undegraded":"2024-03-07T07:20:30.958179+0000","last_fullsized":"2024-03-07T07:20:30.958179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.958087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.958087+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.958087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:43.618103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.454112+0000","last_change":"2024-03-07T07:20:25.454132+0000","last_active":"2024-03-07T07:20:25.454112+0000","last_peered":"2024-03-07T07:20:25.454112+0000","last_clean":"2024-03-07T07:20:25.454112+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:25.454112+0000","last_undegraded":"2024-03-07T07:20:25.454112+0000","last_fullsized":"2024-03-07T07:20:25.454112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.454040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.454040+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.454040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:04:46.690594+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.367621+0000","last_change":"2024-03-07T07:20:35.367640+0000","last_active":"2024-03-07T07:20:35.367621+0000","last_peered":"2024-03-07T07:20:35.367621+0000","last_clean":"2024-03-07T07:20:35.367621+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:35.367621+0000","last_undegraded":"2024-03-07T07:20:35.367621+0000","last_fullsized":"2024-03-07T07:20:35.367621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.367548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.367548+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.367548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:54:24.987586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.536098+0000","last_change":"2024-03-07T07:20:26.536135+0000","last_active":"2024-03-07T07:20:26.536098+0000","last_peered":"2024-03-07T07:20:26.536098+0000","last_clean":"2024-03-07T07:20:26.536098+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:26.536098+0000","last_undegraded":"2024-03-07T07:20:26.536098+0000","last_fullsized":"2024-03-07T07:20:26.536098+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.536002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.536002+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.536002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:53.780258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.446775+0000","last_change":"2024-03-07T07:20:33.446819+0000","last_active":"2024-03-07T07:20:33.446775+0000","last_peered":"2024-03-07T07:20:33.446775+0000","last_clean":"2024-03-07T07:20:33.446775+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:33.446775+0000","last_undegraded":"2024-03-07T07:20:33.446775+0000","last_fullsized":"2024-03-07T07:20:33.446775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.446685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.446685+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.446685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:22:11.407928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.029315+0000","last_change":"2024-03-07T07:20:25.029343+0000","last_active":"2024-03-07T07:20:25.029315+0000","last_peered":"2024-03-07T07:20:25.029315+0000","last_clean":"2024-03-07T07:20:25.029315+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:25.029315+0000","last_undegraded":"2024-03-07T07:20:25.029315+0000","last_fullsized":"2024-03-07T07:20:25.029315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-03-07T07:20:25.029221+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-03-07T07:20:25.029221+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.029221+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:04.169199+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.799734+0000","last_change":"2024-03-07T07:20:35.799774+0000","last_active":"2024-03-07T07:20:35.799734+0000","last_peered":"2024-03-07T07:20:35.799734+0000","last_clean":"2024-03-07T07:20:35.799734+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:35.799734+0000","last_undegraded":"2024-03-07T07:20:35.799734+0000","last_fullsized":"2024-03-07T07:20:35.799734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.799630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.799630+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.799630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:57:58.471370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'10","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.435236+0000","last_change":"2024-03-07T07:20:30.435261+0000","last_active":"2024-03-07T07:20:30.435236+0000","last_peered":"2024-03-07T07:20:30.435236+0000","last_clean":"2024-03-07T07:20:30.435236+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:30.435236+0000","last_undegraded":"2024-03-07T07:20:30.435236+0000","last_fullsized":"2024-03-07T07:20:30.435236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-03-07T07:20:30.435166+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-03-07T07:20:30.435166+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.435166+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:26.716290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7145,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.557533+0000","last_change":"2024-03-07T07:20:30.557553+0000","last_active":"2024-03-07T07:20:30.557533+0000","last_peered":"2024-03-07T07:20:30.557533+0000","last_clean":"2024-03-07T07:20:30.557533+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:30.557533+0000","last_undegraded":"2024-03-07T07:20:30.557533+0000","last_fullsized":"2024-03-07T07:20:30.557533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.557466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.557466+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.557466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:49.526826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.395187+0000","last_change":"2024-03-07T07:20:27.395238+0000","last_active":"2024-03-07T07:20:27.395187+0000","last_peered":"2024-03-07T07:20:27.395187+0000","last_clean":"2024-03-07T07:20:27.395187+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:27.395187+0000","last_undegraded":"2024-03-07T07:20:27.395187+0000","last_fullsized":"2024-03-07T07:20:27.395187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:27.395067+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:27.395067+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.395067+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:54.639323+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:09.516813+0000","last_change":"2024-03-07T07:20:08.037145+0000","last_active":"2024-03-07T07:20:09.516813+0000","last_peered":"2024-03-07T07:20:09.516813+0000","last_clean":"2024-03-07T07:20:09.516813+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:09.516813+0000","last_undegraded":"2024-03-07T07:20:09.516813+0000","last_fullsized":"2024-03-07T07:20:09.516813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:07.005721+0000","last_clean_scrub_stamp":"2024-03-07T07:20:07.005721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.601194+0000","last_change":"2024-03-07T07:20:24.601214+0000","last_active":"2024-03-07T07:20:24.601194+0000","last_peered":"2024-03-07T07:20:24.601194+0000","last_clean":"2024-03-07T07:20:24.601194+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:24.601194+0000","last_undegraded":"2024-03-07T07:20:24.601194+0000","last_fullsized":"2024-03-07T07:20:24.601194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.601115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.601115+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.601115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:51.385830+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.484629+0000","last_change":"2024-03-07T07:20:30.484650+0000","last_active":"2024-03-07T07:20:30.484629+0000","last_peered":"2024-03-07T07:20:30.484629+0000","last_clean":"2024-03-07T07:20:30.484629+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:30.484629+0000","last_undegraded":"2024-03-07T07:20:30.484629+0000","last_fullsized":"2024-03-07T07:20:30.484629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.484538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.484538+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.484538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:07:35.617344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.454535+0000","last_change":"2024-03-07T07:20:26.454556+0000","last_active":"2024-03-07T07:20:26.454535+0000","last_peered":"2024-03-07T07:20:26.454535+0000","last_clean":"2024-03-07T07:20:26.454535+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:26.454535+0000","last_undegraded":"2024-03-07T07:20:26.454535+0000","last_fullsized":"2024-03-07T07:20:26.454535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.454455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.454455+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.454455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:14.787910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.310304+0000","last_change":"2024-03-07T07:20:32.310329+0000","last_active":"2024-03-07T07:20:32.310304+0000","last_peered":"2024-03-07T07:20:32.310304+0000","last_clean":"2024-03-07T07:20:32.310304+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:32.310304+0000","last_undegraded":"2024-03-07T07:20:32.310304+0000","last_fullsized":"2024-03-07T07:20:32.310304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.310206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.310206+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.310206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:59.094921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.468180+0000","last_change":"2024-03-07T07:20:28.468211+0000","last_active":"2024-03-07T07:20:28.468180+0000","last_peered":"2024-03-07T07:20:28.468180+0000","last_clean":"2024-03-07T07:20:28.468180+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:28.468180+0000","last_undegraded":"2024-03-07T07:20:28.468180+0000","last_fullsized":"2024-03-07T07:20:28.468180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-03-07T07:20:28.468107+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-03-07T07:20:28.468107+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.468107+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:47.437467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.192563+0000","last_change":"2024-03-07T07:20:35.192591+0000","last_active":"2024-03-07T07:20:35.192563+0000","last_peered":"2024-03-07T07:20:35.192563+0000","last_clean":"2024-03-07T07:20:35.192563+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:35.192563+0000","last_undegraded":"2024-03-07T07:20:35.192563+0000","last_fullsized":"2024-03-07T07:20:35.192563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.192461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.192461+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.192461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:31:08.707955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.536215+0000","last_change":"2024-03-07T07:20:26.536258+0000","last_active":"2024-03-07T07:20:26.536215+0000","last_peered":"2024-03-07T07:20:26.536215+0000","last_clean":"2024-03-07T07:20:26.536215+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:26.536215+0000","last_undegraded":"2024-03-07T07:20:26.536215+0000","last_fullsized":"2024-03-07T07:20:26.536215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:26.536134+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:26.536134+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.536134+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:46:10.997014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.848503+0000","last_change":"2024-03-07T07:20:33.848526+0000","last_active":"2024-03-07T07:20:33.848503+0000","last_peered":"2024-03-07T07:20:33.848503+0000","last_clean":"2024-03-07T07:20:33.848503+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:33.848503+0000","last_undegraded":"2024-03-07T07:20:33.848503+0000","last_fullsized":"2024-03-07T07:20:33.848503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.848402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.848402+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.848402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:19.906165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.378151+0000","last_change":"2024-03-07T07:20:28.378195+0000","last_active":"2024-03-07T07:20:28.378151+0000","last_peered":"2024-03-07T07:20:28.378151+0000","last_clean":"2024-03-07T07:20:28.378151+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:28.378151+0000","last_undegraded":"2024-03-07T07:20:28.378151+0000","last_fullsized":"2024-03-07T07:20:28.378151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:28.378024+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:28.378024+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.378024+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:43.989608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.518868+0000","last_change":"2024-03-07T07:20:29.518923+0000","last_active":"2024-03-07T07:20:29.518868+0000","last_peered":"2024-03-07T07:20:29.518868+0000","last_clean":"2024-03-07T07:20:29.518868+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:29.518868+0000","last_undegraded":"2024-03-07T07:20:29.518868+0000","last_fullsized":"2024-03-07T07:20:29.518868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.518792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.518792+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.518792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:08.352473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.366239+0000","last_change":"2024-03-07T07:20:26.366272+0000","last_active":"2024-03-07T07:20:26.366239+0000","last_peered":"2024-03-07T07:20:26.366239+0000","last_clean":"2024-03-07T07:20:26.366239+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:26.366239+0000","last_undegraded":"2024-03-07T07:20:26.366239+0000","last_fullsized":"2024-03-07T07:20:26.366239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:26.366132+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:26.366132+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.366132+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:11:00.095407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.493071+0000","last_change":"2024-03-07T07:20:26.493090+0000","last_active":"2024-03-07T07:20:26.493071+0000","last_peered":"2024-03-07T07:20:26.493071+0000","last_clean":"2024-03-07T07:20:26.493071+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:26.493071+0000","last_undegraded":"2024-03-07T07:20:26.493071+0000","last_fullsized":"2024-03-07T07:20:26.493071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.493004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.493004+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.493004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:48:14.950271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.375621+0000","last_change":"2024-03-07T07:20:24.375644+0000","last_active":"2024-03-07T07:20:24.375621+0000","last_peered":"2024-03-07T07:20:24.375621+0000","last_clean":"2024-03-07T07:20:24.375621+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:24.375621+0000","last_undegraded":"2024-03-07T07:20:24.375621+0000","last_fullsized":"2024-03-07T07:20:24.375621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.375526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.375526+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.375526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:48.149456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.460383+0000","last_change":"2024-03-07T07:20:28.460407+0000","last_active":"2024-03-07T07:20:28.460383+0000","last_peered":"2024-03-07T07:20:28.460383+0000","last_clean":"2024-03-07T07:20:28.460383+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:28.460383+0000","last_undegraded":"2024-03-07T07:20:28.460383+0000","last_fullsized":"2024-03-07T07:20:28.460383+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.460298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.460298+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.460298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:35.987508+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.579640+0000","last_change":"2024-03-07T07:20:25.579685+0000","last_active":"2024-03-07T07:20:25.579640+0000","last_peered":"2024-03-07T07:20:25.579640+0000","last_clean":"2024-03-07T07:20:25.579640+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:25.579640+0000","last_undegraded":"2024-03-07T07:20:25.579640+0000","last_fullsized":"2024-03-07T07:20:25.579640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:25.579514+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:25.579514+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.579514+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:22:02.747171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.411709+0000","last_change":"2024-03-07T07:20:29.411730+0000","last_active":"2024-03-07T07:20:29.411709+0000","last_peered":"2024-03-07T07:20:29.411709+0000","last_clean":"2024-03-07T07:20:29.411709+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:29.411709+0000","last_undegraded":"2024-03-07T07:20:29.411709+0000","last_fullsized":"2024-03-07T07:20:29.411709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.411633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.411633+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.411633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:23.085145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.217580+0000","last_change":"2024-03-07T07:20:27.217614+0000","last_active":"2024-03-07T07:20:27.217580+0000","last_peered":"2024-03-07T07:20:27.217580+0000","last_clean":"2024-03-07T07:20:27.217580+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:27.217580+0000","last_undegraded":"2024-03-07T07:20:27.217580+0000","last_fullsized":"2024-03-07T07:20:27.217580+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.217496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.217496+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.217496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:20.891008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.447888+0000","last_change":"2024-03-07T07:20:27.447930+0000","last_active":"2024-03-07T07:20:27.447888+0000","last_peered":"2024-03-07T07:20:27.447888+0000","last_clean":"2024-03-07T07:20:27.447888+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:27.447888+0000","last_undegraded":"2024-03-07T07:20:27.447888+0000","last_fullsized":"2024-03-07T07:20:27.447888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.447800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.447800+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.447800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:12.901646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.295795+0000","last_change":"2024-03-07T07:20:30.295832+0000","last_active":"2024-03-07T07:20:30.295795+0000","last_peered":"2024-03-07T07:20:30.295795+0000","last_clean":"2024-03-07T07:20:30.295795+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:30.295795+0000","last_undegraded":"2024-03-07T07:20:30.295795+0000","last_fullsized":"2024-03-07T07:20:30.295795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.295697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.295697+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.295697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:57.080413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.495363+0000","last_change":"2024-03-07T07:20:28.495387+0000","last_active":"2024-03-07T07:20:28.495363+0000","last_peered":"2024-03-07T07:20:28.495363+0000","last_clean":"2024-03-07T07:20:28.495363+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:28.495363+0000","last_undegraded":"2024-03-07T07:20:28.495363+0000","last_fullsized":"2024-03-07T07:20:28.495363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.495254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.495254+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.495254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:03:13.553501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.024823+0000","last_change":"2024-03-07T07:20:26.024865+0000","last_active":"2024-03-07T07:20:26.024823+0000","last_peered":"2024-03-07T07:20:26.024823+0000","last_clean":"2024-03-07T07:20:26.024823+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:26.024823+0000","last_undegraded":"2024-03-07T07:20:26.024823+0000","last_fullsized":"2024-03-07T07:20:26.024823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.024712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.024712+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.024712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:02.959784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.483953+0000","last_change":"2024-03-07T07:20:25.483990+0000","last_active":"2024-03-07T07:20:25.483953+0000","last_peered":"2024-03-07T07:20:25.483953+0000","last_clean":"2024-03-07T07:20:25.483953+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:25.483953+0000","last_undegraded":"2024-03-07T07:20:25.483953+0000","last_fullsized":"2024-03-07T07:20:25.483953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.483850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.483850+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.483850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:30.756260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.583448+0000","last_change":"2024-03-07T07:20:25.583473+0000","last_active":"2024-03-07T07:20:25.583448+0000","last_peered":"2024-03-07T07:20:25.583448+0000","last_clean":"2024-03-07T07:20:25.583448+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:25.583448+0000","last_undegraded":"2024-03-07T07:20:25.583448+0000","last_fullsized":"2024-03-07T07:20:25.583448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:25.583365+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:25.583365+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.583365+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:04.723343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.875907+0000","last_change":"2024-03-07T07:20:32.875930+0000","last_active":"2024-03-07T07:20:32.875907+0000","last_peered":"2024-03-07T07:20:32.875907+0000","last_clean":"2024-03-07T07:20:32.875907+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:32.875907+0000","last_undegraded":"2024-03-07T07:20:32.875907+0000","last_fullsized":"2024-03-07T07:20:32.875907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.875816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.875816+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.875816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:12.691716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'3","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.564515+0000","last_change":"2024-03-07T07:20:24.564564+0000","last_active":"2024-03-07T07:20:24.564515+0000","last_peered":"2024-03-07T07:20:24.564515+0000","last_clean":"2024-03-07T07:20:24.564515+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:24.564515+0000","last_undegraded":"2024-03-07T07:20:24.564515+0000","last_fullsized":"2024-03-07T07:20:24.564515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-03-07T07:20:24.564421+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-03-07T07:20:24.564421+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.564421+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:18.237933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.565219+0000","last_change":"2024-03-07T07:20:30.565245+0000","last_active":"2024-03-07T07:20:30.565219+0000","last_peered":"2024-03-07T07:20:30.565219+0000","last_clean":"2024-03-07T07:20:30.565219+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:30.565219+0000","last_undegraded":"2024-03-07T07:20:30.565219+0000","last_fullsized":"2024-03-07T07:20:30.565219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.565130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.565130+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.565130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:09.705108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:22.561419+0000","last_change":"2024-03-07T07:20:22.561437+0000","last_active":"2024-03-07T07:20:22.561419+0000","last_peered":"2024-03-07T07:20:22.561419+0000","last_clean":"2024-03-07T07:20:22.561419+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:22.561419+0000","last_undegraded":"2024-03-07T07:20:22.561419+0000","last_fullsized":"2024-03-07T07:20:22.561419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:22.561336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:22.561336+0000","last_clean_scrub_stamp":"2024-03-07T07:20:22.561336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:13:14.464836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.219191+0000","last_change":"2024-03-07T07:20:28.219227+0000","last_active":"2024-03-07T07:20:28.219191+0000","last_peered":"2024-03-07T07:20:28.219191+0000","last_clean":"2024-03-07T07:20:28.219191+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:28.219191+0000","last_undegraded":"2024-03-07T07:20:28.219191+0000","last_fullsized":"2024-03-07T07:20:28.219191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.219041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.219041+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.219041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:11:01.948316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:22.598033+0000","last_change":"2024-03-07T07:20:22.598054+0000","last_active":"2024-03-07T07:20:22.598033+0000","last_peered":"2024-03-07T07:20:22.598033+0000","last_clean":"2024-03-07T07:20:22.598033+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:22.598033+0000","last_undegraded":"2024-03-07T07:20:22.598033+0000","last_fullsized":"2024-03-07T07:20:22.598033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:22.597947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:22.597947+0000","last_clean_scrub_stamp":"2024-03-07T07:20:22.597947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:08.051793+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.582226+0000","last_change":"2024-03-07T07:20:27.582267+0000","last_active":"2024-03-07T07:20:27.582226+0000","last_peered":"2024-03-07T07:20:27.582226+0000","last_clean":"2024-03-07T07:20:27.582226+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:27.582226+0000","last_undegraded":"2024-03-07T07:20:27.582226+0000","last_fullsized":"2024-03-07T07:20:27.582226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.582148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.582148+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.582148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:13:19.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:23.596312+0000","last_change":"2024-03-07T07:20:23.596350+0000","last_active":"2024-03-07T07:20:23.596312+0000","last_peered":"2024-03-07T07:20:23.596312+0000","last_clean":"2024-03-07T07:20:23.596312+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:23.596312+0000","last_undegraded":"2024-03-07T07:20:23.596312+0000","last_fullsized":"2024-03-07T07:20:23.596312+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:23.596186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:23.596186+0000","last_clean_scrub_stamp":"2024-03-07T07:20:23.596186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:59.634407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:36.360741+0000","last_change":"2024-03-07T07:20:36.360759+0000","last_active":"2024-03-07T07:20:36.360741+0000","last_peered":"2024-03-07T07:20:36.360741+0000","last_clean":"2024-03-07T07:20:36.360741+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:36.360741+0000","last_undegraded":"2024-03-07T07:20:36.360741+0000","last_fullsized":"2024-03-07T07:20:36.360741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:36.360672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:36.360672+0000","last_clean_scrub_stamp":"2024-03-07T07:20:36.360672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:31:37.413085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.355004+0000","last_change":"2024-03-07T07:20:25.355025+0000","last_active":"2024-03-07T07:20:25.355004+0000","last_peered":"2024-03-07T07:20:25.355004+0000","last_clean":"2024-03-07T07:20:25.355004+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:25.355004+0000","last_undegraded":"2024-03-07T07:20:25.355004+0000","last_fullsized":"2024-03-07T07:20:25.355004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.354911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.354911+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.354911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:19.028423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:37.170992+0000","last_change":"2024-03-07T07:20:37.171042+0000","last_active":"2024-03-07T07:20:37.170992+0000","last_peered":"2024-03-07T07:20:37.170992+0000","last_clean":"2024-03-07T07:20:37.170992+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:37.170992+0000","last_undegraded":"2024-03-07T07:20:37.170992+0000","last_fullsized":"2024-03-07T07:20:37.170992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:37.170881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:37.170881+0000","last_clean_scrub_stamp":"2024-03-07T07:20:37.170881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:39.778472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.419580+0000","last_change":"2024-03-07T07:20:31.419619+0000","last_active":"2024-03-07T07:20:31.419580+0000","last_peered":"2024-03-07T07:20:31.419580+0000","last_clean":"2024-03-07T07:20:31.419580+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:31.419580+0000","last_undegraded":"2024-03-07T07:20:31.419580+0000","last_fullsized":"2024-03-07T07:20:31.419580+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.419472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.419472+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.419472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:11.235371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.433685+0000","last_change":"2024-03-07T07:20:33.433711+0000","last_active":"2024-03-07T07:20:33.433685+0000","last_peered":"2024-03-07T07:20:33.433685+0000","last_clean":"2024-03-07T07:20:33.433685+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:33.433685+0000","last_undegraded":"2024-03-07T07:20:33.433685+0000","last_fullsized":"2024-03-07T07:20:33.433685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.433555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.433555+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.433555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:13.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.263700+0000","last_change":"2024-03-07T07:20:25.263739+0000","last_active":"2024-03-07T07:20:25.263700+0000","last_peered":"2024-03-07T07:20:25.263700+0000","last_clean":"2024-03-07T07:20:25.263700+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:25.263700+0000","last_undegraded":"2024-03-07T07:20:25.263700+0000","last_fullsized":"2024-03-07T07:20:25.263700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.263585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.263585+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.263585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:33.629765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"33'119","reported_seq":204,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.434174+0000","last_change":"2024-03-07T07:20:25.434236+0000","last_active":"2024-03-07T07:20:25.434174+0000","last_peered":"2024-03-07T07:20:25.434174+0000","last_clean":"2024-03-07T07:20:25.434174+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:25.434174+0000","last_undegraded":"2024-03-07T07:20:25.434174+0000","last_fullsized":"2024-03-07T07:20:25.434174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2024-03-07T07:20:25.434050+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2024-03-07T07:20:25.434050+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.434050+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:04.574028+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":7943,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":15886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:20:46.014 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:46.014 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:46.014 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:46.015 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-03-07T07:20:07.005721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=7, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=7, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T07:20:46.015 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-07T07:20:46.551 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:20:46.554 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:21:06.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T07:21:06.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.118+0000 7f23e7710640 1 Processor -- start 2024-03-07T07:21:06.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.118+0000 7f23e7710640 1 -- start start 2024-03-07T07:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e7710640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f23e0144540 0x7f23e0162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e7710640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23e0162ab0 0x7f23e0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e7710640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00b4bd0 con 0x7f23e00b5dd0 2024-03-07T07:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e7710640 1 -- --> v2:172.21.15.64:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e0141700 con 0x7f23e0162ab0 2024-03-07T07:21:06.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e7710640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e0161e00 con 0x7f23e0144540 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e5cad640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23e0162ab0 0x7f23e0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e5cad640 1 --2- >> v2:172.21.15.64:3300/0 conn(0x7f23e0162ab0 0x7f23e0071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33130/0 (socket says 172.21.15.64:33130) 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e5cad640 1 -- 172.21.15.64:0/249695977 learned_addr learned my addr 172.21.15.64:0/249695977 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.119+0000 7f23e4cab640 1 --2- 172.21.15.64:0/249695977 >> v2:172.21.15.120:3300/0 conn(0x7f23e0144540 0x7f23e0162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.120+0000 7f23e5cad640 1 -- 172.21.15.64:0/249695977 >> v2:172.21.15.120:3300/0 conn(0x7f23e0144540 msgr2=0x7f23e0162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.120+0000 7f23e5cad640 1 --2- 172.21.15.64:0/249695977 >> v2:172.21.15.120:3300/0 conn(0x7f23e0144540 0x7f23e0162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.120+0000 7f23e5cad640 1 -- 172.21.15.64:0/249695977 >> v1:172.21.15.120:6789/0 conn(0x7f23e00b5dd0 legacy=0x7f23e00b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.120+0000 7f23e5cad640 1 -- 172.21.15.64:0/249695977 --> v2:172.21.15.64:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00b4eb0 con 0x7f23e0162ab0 2024-03-07T07:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23e5cad640 1 --2- 172.21.15.64:0/249695977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e0162ab0 0x7f23e0071410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23e008d440 tx=0x7f23d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa4568d0bcefefd server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23d7fff640 1 -- 172.21.15.64:0/249695977 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8035020 con 0x7f23e0162ab0 2024-03-07T07:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23d7fff640 1 -- 172.21.15.64:0/249695977 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d8031870 con 0x7f23e0162ab0 2024-03-07T07:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23d7fff640 1 -- 172.21.15.64:0/249695977 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8031c40 con 0x7f23e0162ab0 2024-03-07T07:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23e7710640 1 -- 172.21.15.64:0/249695977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e0162ab0 msgr2=0x7f23e0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23e7710640 1 --2- 172.21.15.64:0/249695977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e0162ab0 0x7f23e0071410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23e008d440 tx=0x7f23d802b910 comp rx=0 tx=0).stop 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23e7710640 1 -- 172.21.15.64:0/249695977 shutdown_connections 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23e7710640 1 --2- 172.21.15.64:0/249695977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e0162ab0 0x7f23e0071410 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23e7710640 1 --2- 172.21.15.64:0/249695977 >> v2:172.21.15.120:3300/0 conn(0x7f23e0144540 0x7f23e0162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.121+0000 7f23e7710640 1 -- 172.21.15.64:0/249695977 >> 172.21.15.64:0/249695977 conn(0x7f23e00d0e70 msgr2=0x7f23e00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.122+0000 7f23e7710640 1 -- 172.21.15.64:0/249695977 shutdown_connections 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.122+0000 7f23e7710640 1 -- 172.21.15.64:0/249695977 wait complete. 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 Processor -- start 2024-03-07T07:21:06.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 -- start start 2024-03-07T07:21:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e00b5dd0 0x7f23e013ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23e0144540 0x7f23e014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23e014d670 0x7f23e015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b61b0 con 0x7f23e00b5dd0 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0162050 con 0x7f23e0144540 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.123+0000 7f23e7710640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0071950 con 0x7f23e014d670 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e00b5dd0 0x7f23e013ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e00b5dd0 0x7f23e013ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33134/0 (socket says 172.21.15.64:33134) 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 -- 172.21.15.64:0/674689806 learned_addr learned my addr 172.21.15.64:0/674689806 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e4cab640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23e0144540 0x7f23e014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e5cad640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23e014d670 0x7f23e015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 -- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23e014d670 msgr2=0x7f23e015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23e014d670 0x7f23e015b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 -- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23e0144540 msgr2=0x7f23e014ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23e0144540 0x7f23e014ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 -- 172.21.15.64:0/674689806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0141da0 con 0x7f23e00b5dd0 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e5cad640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23e014d670 0x7f23e015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e54ac640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e00b5dd0 0x7f23e013ccc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00c790 tx=0x7f23dc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23d5ffb640 1 -- 172.21.15.64:0/674689806 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc016020 con 0x7f23e00b5dd0 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23d5ffb640 1 -- 172.21.15.64:0/674689806 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc006d50 con 0x7f23e00b5dd0 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e0142080 con 0x7f23e00b5dd0 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23d5ffb640 1 -- 172.21.15.64:0/674689806 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc003bb0 con 0x7f23e00b5dd0 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.124+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e0142640 con 0x7f23e00b5dd0 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.125+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e0062c90 con 0x7f23e00b5dd0 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.126+0000 7f23d5ffb640 1 -- 172.21.15.64:0/674689806 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f23dc00a850 con 0x7f23e00b5dd0 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.126+0000 7f23d5ffb640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.126+0000 7f23d5ffb640 1 -- 172.21.15.64:0/674689806 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23dc089e50 con 0x7f23e00b5dd0 2024-03-07T07:21:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.127+0000 7f23e4cab640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.127+0000 7f23e4cab640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f23e01595d0 tx=0x7f23d0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.131+0000 7f23d5ffb640 1 -- 172.21.15.64:0/674689806 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f23dc020020 con 0x7f23e00b5dd0 2024-03-07T07:21:06.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.309+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 --> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23e0071d20 con 0x7f23c406b4d0 2024-03-07T07:21:06.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.315+0000 7f23d5ffb640 1 -- 172.21.15.64:0/674689806 <== mgr.4105 v2:172.21.15.120:6832/182409167 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251889 (secure 0 0 0) 0x7f23e0071d20 con 0x7f23c406b4d0 2024-03-07T07:21:06.319 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:21:06.320 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-03-07T07:21:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 msgr2=0x7f23c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f23e01595d0 tx=0x7f23d0009190 comp rx=0 tx=0).stop 2024-03-07T07:21:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e00b5dd0 msgr2=0x7f23e013ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e00b5dd0 0x7f23e013ccc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00c790 tx=0x7f23dc00cc50 comp rx=0 tx=0).stop 2024-03-07T07:21:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 shutdown_connections 2024-03-07T07:21:06.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f23c406b4d0 0x7f23c406d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23e014d670 0x7f23e015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f23e0144540 0x7f23e014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 --2- 172.21.15.64:0/674689806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e00b5dd0 0x7f23e013ccc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 >> 172.21.15.64:0/674689806 conn(0x7f23e00d0e70 msgr2=0x7f23e015e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 shutdown_connections 2024-03-07T07:21:06.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.318+0000 7f23e7710640 1 -- 172.21.15.64:0/674689806 wait complete. 2024-03-07T07:21:06.330 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-03-07T07:21:06.203955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598311,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220272,"kb_used_data":4144,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754448,"statfs":{"total":773094113280,"available":772868554752,"internally_reserved":0,"allocated":4243456,"data_stored":2369550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003236"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.270024+0000","last_change":"2024-03-07T07:20:30.270046+0000","last_active":"2024-03-07T07:20:30.270024+0000","last_peered":"2024-03-07T07:20:30.270024+0000","last_clean":"2024-03-07T07:20:30.270024+0000","last_became_active":"2024-03-07T07:20:08.025146+0000","last_became_peered":"2024-03-07T07:20:08.025146+0000","last_unstale":"2024-03-07T07:20:30.270024+0000","last_undegraded":"2024-03-07T07:20:30.270024+0000","last_fullsized":"2024-03-07T07:20:30.270024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.269932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.269932+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.269932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:45.881517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:38.302425+0000","last_change":"2024-03-07T07:20:38.302447+0000","last_active":"2024-03-07T07:20:38.302425+0000","last_peered":"2024-03-07T07:20:38.302425+0000","last_clean":"2024-03-07T07:20:38.302425+0000","last_became_active":"2024-03-07T07:20:08.036422+0000","last_became_peered":"2024-03-07T07:20:08.036422+0000","last_unstale":"2024-03-07T07:20:38.302425+0000","last_undegraded":"2024-03-07T07:20:38.302425+0000","last_fullsized":"2024-03-07T07:20:38.302425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:38.302352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:38.302352+0000","last_clean_scrub_stamp":"2024-03-07T07:20:38.302352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:32:29.833779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.821680+0000","last_change":"2024-03-07T07:20:34.821726+0000","last_active":"2024-03-07T07:20:34.821680+0000","last_peered":"2024-03-07T07:20:34.821680+0000","last_clean":"2024-03-07T07:20:34.821680+0000","last_became_active":"2024-03-07T07:20:08.037683+0000","last_became_peered":"2024-03-07T07:20:08.037683+0000","last_unstale":"2024-03-07T07:20:34.821680+0000","last_undegraded":"2024-03-07T07:20:34.821680+0000","last_fullsized":"2024-03-07T07:20:34.821680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.821575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.821575+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.821575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:28:27.159052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.507728+0000","last_change":"2024-03-07T07:20:32.507769+0000","last_active":"2024-03-07T07:20:32.507728+0000","last_peered":"2024-03-07T07:20:32.507728+0000","last_clean":"2024-03-07T07:20:32.507728+0000","last_became_active":"2024-03-07T07:20:08.042176+0000","last_became_peered":"2024-03-07T07:20:08.042176+0000","last_unstale":"2024-03-07T07:20:32.507728+0000","last_undegraded":"2024-03-07T07:20:32.507728+0000","last_fullsized":"2024-03-07T07:20:32.507728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.507639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.507639+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.507639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:28.788762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.361831+0000","last_change":"2024-03-07T07:20:31.361869+0000","last_active":"2024-03-07T07:20:31.361831+0000","last_peered":"2024-03-07T07:20:31.361831+0000","last_clean":"2024-03-07T07:20:31.361831+0000","last_became_active":"2024-03-07T07:20:08.037576+0000","last_became_peered":"2024-03-07T07:20:08.037576+0000","last_unstale":"2024-03-07T07:20:31.361831+0000","last_undegraded":"2024-03-07T07:20:31.361831+0000","last_fullsized":"2024-03-07T07:20:31.361831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.361723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.361723+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.361723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:23:38.499288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.503767+0000","last_change":"2024-03-07T07:20:24.503806+0000","last_active":"2024-03-07T07:20:24.503767+0000","last_peered":"2024-03-07T07:20:24.503767+0000","last_clean":"2024-03-07T07:20:24.503767+0000","last_became_active":"2024-03-07T07:20:08.027887+0000","last_became_peered":"2024-03-07T07:20:08.027887+0000","last_unstale":"2024-03-07T07:20:24.503767+0000","last_undegraded":"2024-03-07T07:20:24.503767+0000","last_fullsized":"2024-03-07T07:20:24.503767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.503653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.503653+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.503653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:46:08.964533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.460317+0000","last_change":"2024-03-07T07:20:27.460341+0000","last_active":"2024-03-07T07:20:27.460317+0000","last_peered":"2024-03-07T07:20:27.460317+0000","last_clean":"2024-03-07T07:20:27.460317+0000","last_became_active":"2024-03-07T07:20:08.032034+0000","last_became_peered":"2024-03-07T07:20:08.032034+0000","last_unstale":"2024-03-07T07:20:27.460317+0000","last_undegraded":"2024-03-07T07:20:27.460317+0000","last_fullsized":"2024-03-07T07:20:27.460317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.460227+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:09:10.521273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.412734+0000","last_change":"2024-03-07T07:20:34.412773+0000","last_active":"2024-03-07T07:20:34.412734+0000","last_peered":"2024-03-07T07:20:34.412734+0000","last_clean":"2024-03-07T07:20:34.412734+0000","last_became_active":"2024-03-07T07:20:08.035218+0000","last_became_peered":"2024-03-07T07:20:08.035218+0000","last_unstale":"2024-03-07T07:20:34.412734+0000","last_undegraded":"2024-03-07T07:20:34.412734+0000","last_fullsized":"2024-03-07T07:20:34.412734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.412603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.412603+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.412603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:20.470367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.235483+0000","last_change":"2024-03-07T07:20:34.235509+0000","last_active":"2024-03-07T07:20:34.235483+0000","last_peered":"2024-03-07T07:20:34.235483+0000","last_clean":"2024-03-07T07:20:34.235483+0000","last_became_active":"2024-03-07T07:20:08.033741+0000","last_became_peered":"2024-03-07T07:20:08.033741+0000","last_unstale":"2024-03-07T07:20:34.235483+0000","last_undegraded":"2024-03-07T07:20:34.235483+0000","last_fullsized":"2024-03-07T07:20:34.235483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.235381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.235381+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.235381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:03:19.293627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.533766+0000","last_change":"2024-03-07T07:20:31.533793+0000","last_active":"2024-03-07T07:20:31.533766+0000","last_peered":"2024-03-07T07:20:31.533766+0000","last_clean":"2024-03-07T07:20:31.533766+0000","last_became_active":"2024-03-07T07:20:08.036980+0000","last_became_peered":"2024-03-07T07:20:08.036980+0000","last_unstale":"2024-03-07T07:20:31.533766+0000","last_undegraded":"2024-03-07T07:20:31.533766+0000","last_fullsized":"2024-03-07T07:20:31.533766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.533698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.533698+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.533698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:44.193713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:36.475255+0000","last_change":"2024-03-07T07:20:36.475278+0000","last_active":"2024-03-07T07:20:36.475255+0000","last_peered":"2024-03-07T07:20:36.475255+0000","last_clean":"2024-03-07T07:20:36.475255+0000","last_became_active":"2024-03-07T07:20:08.039422+0000","last_became_peered":"2024-03-07T07:20:08.039422+0000","last_unstale":"2024-03-07T07:20:36.475255+0000","last_undegraded":"2024-03-07T07:20:36.475255+0000","last_fullsized":"2024-03-07T07:20:36.475255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:36.475163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:36.475163+0000","last_clean_scrub_stamp":"2024-03-07T07:20:36.475163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:57:59.146903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.974491+0000","last_change":"2024-03-07T07:20:29.974546+0000","last_active":"2024-03-07T07:20:29.974491+0000","last_peered":"2024-03-07T07:20:29.974491+0000","last_clean":"2024-03-07T07:20:29.974491+0000","last_became_active":"2024-03-07T07:20:08.027961+0000","last_became_peered":"2024-03-07T07:20:08.027961+0000","last_unstale":"2024-03-07T07:20:29.974491+0000","last_undegraded":"2024-03-07T07:20:29.974491+0000","last_fullsized":"2024-03-07T07:20:29.974491+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.974395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.974395+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.974395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:07:35.107200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.363449+0000","last_change":"2024-03-07T07:20:30.363468+0000","last_active":"2024-03-07T07:20:30.363449+0000","last_peered":"2024-03-07T07:20:30.363449+0000","last_clean":"2024-03-07T07:20:30.363449+0000","last_became_active":"2024-03-07T07:20:08.033336+0000","last_became_peered":"2024-03-07T07:20:08.033336+0000","last_unstale":"2024-03-07T07:20:30.363449+0000","last_undegraded":"2024-03-07T07:20:30.363449+0000","last_fullsized":"2024-03-07T07:20:30.363449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.363368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.363368+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.363368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:11:04.092642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.408466+0000","last_change":"2024-03-07T07:20:32.408489+0000","last_active":"2024-03-07T07:20:32.408466+0000","last_peered":"2024-03-07T07:20:32.408466+0000","last_clean":"2024-03-07T07:20:32.408466+0000","last_became_active":"2024-03-07T07:20:08.021679+0000","last_became_peered":"2024-03-07T07:20:08.021679+0000","last_unstale":"2024-03-07T07:20:32.408466+0000","last_undegraded":"2024-03-07T07:20:32.408466+0000","last_fullsized":"2024-03-07T07:20:32.408466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.408389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.408389+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.408389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:18.466153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.048120+0000","last_change":"2024-03-07T07:20:27.048143+0000","last_active":"2024-03-07T07:20:27.048120+0000","last_peered":"2024-03-07T07:20:27.048120+0000","last_clean":"2024-03-07T07:20:27.048120+0000","last_became_active":"2024-03-07T07:20:08.020852+0000","last_became_peered":"2024-03-07T07:20:08.020852+0000","last_unstale":"2024-03-07T07:20:27.048120+0000","last_undegraded":"2024-03-07T07:20:27.048120+0000","last_fullsized":"2024-03-07T07:20:27.048120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.047987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.047987+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.047987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:09:10.109033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.438565+0000","last_change":"2024-03-07T07:20:35.438587+0000","last_active":"2024-03-07T07:20:35.438565+0000","last_peered":"2024-03-07T07:20:35.438565+0000","last_clean":"2024-03-07T07:20:35.438565+0000","last_became_active":"2024-03-07T07:20:08.035134+0000","last_became_peered":"2024-03-07T07:20:08.035134+0000","last_unstale":"2024-03-07T07:20:35.438565+0000","last_undegraded":"2024-03-07T07:20:35.438565+0000","last_fullsized":"2024-03-07T07:20:35.438565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.438466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.438466+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.438466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:28:27.775943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.435073+0000","last_change":"2024-03-07T07:20:32.435100+0000","last_active":"2024-03-07T07:20:32.435073+0000","last_peered":"2024-03-07T07:20:32.435073+0000","last_clean":"2024-03-07T07:20:32.435073+0000","last_became_active":"2024-03-07T07:20:08.033216+0000","last_became_peered":"2024-03-07T07:20:08.033216+0000","last_unstale":"2024-03-07T07:20:32.435073+0000","last_undegraded":"2024-03-07T07:20:32.435073+0000","last_fullsized":"2024-03-07T07:20:32.435073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.434962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.434962+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.434962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:44:38.521587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:37.349723+0000","last_change":"2024-03-07T07:20:37.349750+0000","last_active":"2024-03-07T07:20:37.349723+0000","last_peered":"2024-03-07T07:20:37.349723+0000","last_clean":"2024-03-07T07:20:37.349723+0000","last_became_active":"2024-03-07T07:20:08.035131+0000","last_became_peered":"2024-03-07T07:20:08.035131+0000","last_unstale":"2024-03-07T07:20:37.349723+0000","last_undegraded":"2024-03-07T07:20:37.349723+0000","last_fullsized":"2024-03-07T07:20:37.349723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:37.349647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:37.349647+0000","last_clean_scrub_stamp":"2024-03-07T07:20:37.349647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:20:08.492778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:40.291709+0000","last_change":"2024-03-07T07:20:40.291727+0000","last_active":"2024-03-07T07:20:40.291709+0000","last_peered":"2024-03-07T07:20:40.291709+0000","last_clean":"2024-03-07T07:20:40.291709+0000","last_became_active":"2024-03-07T07:20:08.037643+0000","last_became_peered":"2024-03-07T07:20:08.037643+0000","last_unstale":"2024-03-07T07:20:40.291709+0000","last_undegraded":"2024-03-07T07:20:40.291709+0000","last_fullsized":"2024-03-07T07:20:40.291709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:40.291636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:40.291636+0000","last_clean_scrub_stamp":"2024-03-07T07:20:40.291636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:37:49.673602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.446127+0000","last_change":"2024-03-07T07:20:24.446147+0000","last_active":"2024-03-07T07:20:24.446127+0000","last_peered":"2024-03-07T07:20:24.446127+0000","last_clean":"2024-03-07T07:20:24.446127+0000","last_became_active":"2024-03-07T07:20:08.025197+0000","last_became_peered":"2024-03-07T07:20:08.025197+0000","last_unstale":"2024-03-07T07:20:24.446127+0000","last_undegraded":"2024-03-07T07:20:24.446127+0000","last_fullsized":"2024-03-07T07:20:24.446127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.446044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.446044+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.446044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:39:01.221698+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:39.234278+0000","last_change":"2024-03-07T07:20:39.234308+0000","last_active":"2024-03-07T07:20:39.234278+0000","last_peered":"2024-03-07T07:20:39.234278+0000","last_clean":"2024-03-07T07:20:39.234278+0000","last_became_active":"2024-03-07T07:20:08.039571+0000","last_became_peered":"2024-03-07T07:20:08.039571+0000","last_unstale":"2024-03-07T07:20:39.234278+0000","last_undegraded":"2024-03-07T07:20:39.234278+0000","last_fullsized":"2024-03-07T07:20:39.234278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:39.234165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:39.234165+0000","last_clean_scrub_stamp":"2024-03-07T07:20:39.234165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:19.050064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:36.147437+0000","last_change":"2024-03-07T07:20:36.147464+0000","last_active":"2024-03-07T07:20:36.147437+0000","last_peered":"2024-03-07T07:20:36.147437+0000","last_clean":"2024-03-07T07:20:36.147437+0000","last_became_active":"2024-03-07T07:20:08.039488+0000","last_became_peered":"2024-03-07T07:20:08.039488+0000","last_unstale":"2024-03-07T07:20:36.147437+0000","last_undegraded":"2024-03-07T07:20:36.147437+0000","last_fullsized":"2024-03-07T07:20:36.147437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:36.147347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:36.147347+0000","last_clean_scrub_stamp":"2024-03-07T07:20:36.147347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:07:41.280152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.551867+0000","last_change":"2024-03-07T07:20:29.551890+0000","last_active":"2024-03-07T07:20:29.551867+0000","last_peered":"2024-03-07T07:20:29.551867+0000","last_clean":"2024-03-07T07:20:29.551867+0000","last_became_active":"2024-03-07T07:20:08.031049+0000","last_became_peered":"2024-03-07T07:20:08.031049+0000","last_unstale":"2024-03-07T07:20:29.551867+0000","last_undegraded":"2024-03-07T07:20:29.551867+0000","last_fullsized":"2024-03-07T07:20:29.551867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.551797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.551797+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.551797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:56.336512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.003735+0000","last_change":"2024-03-07T07:20:29.003759+0000","last_active":"2024-03-07T07:20:29.003735+0000","last_peered":"2024-03-07T07:20:29.003735+0000","last_clean":"2024-03-07T07:20:29.003735+0000","last_became_active":"2024-03-07T07:20:08.023848+0000","last_became_peered":"2024-03-07T07:20:08.023848+0000","last_unstale":"2024-03-07T07:20:29.003735+0000","last_undegraded":"2024-03-07T07:20:29.003735+0000","last_fullsized":"2024-03-07T07:20:29.003735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.003642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.003642+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.003642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:31:02.519136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.497765+0000","last_change":"2024-03-07T07:20:28.497801+0000","last_active":"2024-03-07T07:20:28.497765+0000","last_peered":"2024-03-07T07:20:28.497765+0000","last_clean":"2024-03-07T07:20:28.497765+0000","last_became_active":"2024-03-07T07:20:08.032580+0000","last_became_peered":"2024-03-07T07:20:08.032580+0000","last_unstale":"2024-03-07T07:20:28.497765+0000","last_undegraded":"2024-03-07T07:20:28.497765+0000","last_fullsized":"2024-03-07T07:20:28.497765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.497661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.497661+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.497661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:16.831116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.416477+0000","last_change":"2024-03-07T07:20:34.416506+0000","last_active":"2024-03-07T07:20:34.416477+0000","last_peered":"2024-03-07T07:20:34.416477+0000","last_clean":"2024-03-07T07:20:34.416477+0000","last_became_active":"2024-03-07T07:20:08.031278+0000","last_became_peered":"2024-03-07T07:20:08.031278+0000","last_unstale":"2024-03-07T07:20:34.416477+0000","last_undegraded":"2024-03-07T07:20:34.416477+0000","last_fullsized":"2024-03-07T07:20:34.416477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.416399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.416399+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.416399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:13:02.792116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:34.493156+0000","last_change":"2024-03-07T07:20:34.493193+0000","last_active":"2024-03-07T07:20:34.493156+0000","last_peered":"2024-03-07T07:20:34.493156+0000","last_clean":"2024-03-07T07:20:34.493156+0000","last_became_active":"2024-03-07T07:20:08.043362+0000","last_became_peered":"2024-03-07T07:20:08.043362+0000","last_unstale":"2024-03-07T07:20:34.493156+0000","last_undegraded":"2024-03-07T07:20:34.493156+0000","last_fullsized":"2024-03-07T07:20:34.493156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:34.493061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:34.493061+0000","last_clean_scrub_stamp":"2024-03-07T07:20:34.493061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:20.550825+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.035852+0000","last_change":"2024-03-07T07:20:28.035891+0000","last_active":"2024-03-07T07:20:28.035852+0000","last_peered":"2024-03-07T07:20:28.035852+0000","last_clean":"2024-03-07T07:20:28.035852+0000","last_became_active":"2024-03-07T07:20:08.021709+0000","last_became_peered":"2024-03-07T07:20:08.021709+0000","last_unstale":"2024-03-07T07:20:28.035852+0000","last_undegraded":"2024-03-07T07:20:28.035852+0000","last_fullsized":"2024-03-07T07:20:28.035852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.035749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.035749+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.035749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:16.369204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:40.221262+0000","last_change":"2024-03-07T07:20:40.221302+0000","last_active":"2024-03-07T07:20:40.221262+0000","last_peered":"2024-03-07T07:20:40.221262+0000","last_clean":"2024-03-07T07:20:40.221262+0000","last_became_active":"2024-03-07T07:20:08.040684+0000","last_became_peered":"2024-03-07T07:20:08.040684+0000","last_unstale":"2024-03-07T07:20:40.221262+0000","last_undegraded":"2024-03-07T07:20:40.221262+0000","last_fullsized":"2024-03-07T07:20:40.221262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:40.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:40.221142+0000","last_clean_scrub_stamp":"2024-03-07T07:20:40.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:26.278906+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.458572+0000","last_change":"2024-03-07T07:20:26.458614+0000","last_active":"2024-03-07T07:20:26.458572+0000","last_peered":"2024-03-07T07:20:26.458572+0000","last_clean":"2024-03-07T07:20:26.458572+0000","last_became_active":"2024-03-07T07:20:08.031336+0000","last_became_peered":"2024-03-07T07:20:08.031336+0000","last_unstale":"2024-03-07T07:20:26.458572+0000","last_undegraded":"2024-03-07T07:20:26.458572+0000","last_fullsized":"2024-03-07T07:20:26.458572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.458474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.458474+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.458474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:03.393547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.254635+0000","last_change":"2024-03-07T07:20:29.254656+0000","last_active":"2024-03-07T07:20:29.254635+0000","last_peered":"2024-03-07T07:20:29.254635+0000","last_clean":"2024-03-07T07:20:29.254635+0000","last_became_active":"2024-03-07T07:20:08.023083+0000","last_became_peered":"2024-03-07T07:20:08.023083+0000","last_unstale":"2024-03-07T07:20:29.254635+0000","last_undegraded":"2024-03-07T07:20:29.254635+0000","last_fullsized":"2024-03-07T07:20:29.254635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.254540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.254540+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.254540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:56.498796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.493976+0000","last_change":"2024-03-07T07:20:29.493999+0000","last_active":"2024-03-07T07:20:29.493976+0000","last_peered":"2024-03-07T07:20:29.493976+0000","last_clean":"2024-03-07T07:20:29.493976+0000","last_became_active":"2024-03-07T07:20:08.033541+0000","last_became_peered":"2024-03-07T07:20:08.033541+0000","last_unstale":"2024-03-07T07:20:29.493976+0000","last_undegraded":"2024-03-07T07:20:29.493976+0000","last_fullsized":"2024-03-07T07:20:29.493976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.493902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.493902+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.493902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:31:03.009396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.538255+0000","last_change":"2024-03-07T07:20:28.538294+0000","last_active":"2024-03-07T07:20:28.538255+0000","last_peered":"2024-03-07T07:20:28.538255+0000","last_clean":"2024-03-07T07:20:28.538255+0000","last_became_active":"2024-03-07T07:20:08.030259+0000","last_became_peered":"2024-03-07T07:20:08.030259+0000","last_unstale":"2024-03-07T07:20:28.538255+0000","last_undegraded":"2024-03-07T07:20:28.538255+0000","last_fullsized":"2024-03-07T07:20:28.538255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.538127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.538127+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.538127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:05:04.576347+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.260297+0000","last_change":"2024-03-07T07:20:26.260329+0000","last_active":"2024-03-07T07:20:26.260297+0000","last_peered":"2024-03-07T07:20:26.260297+0000","last_clean":"2024-03-07T07:20:26.260297+0000","last_became_active":"2024-03-07T07:20:07.025071+0000","last_became_peered":"2024-03-07T07:20:07.025071+0000","last_unstale":"2024-03-07T07:20:26.260297+0000","last_undegraded":"2024-03-07T07:20:26.260297+0000","last_fullsized":"2024-03-07T07:20:26.260297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'4","last_scrub_stamp":"2024-03-07T07:20:26.260206+0000","last_deep_scrub":"32'4","last_deep_scrub_stamp":"2024-03-07T07:20:26.260206+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.260206+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:50.034136+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.275374+0000","last_change":"2024-03-07T07:20:31.275409+0000","last_active":"2024-03-07T07:20:31.275374+0000","last_peered":"2024-03-07T07:20:31.275374+0000","last_clean":"2024-03-07T07:20:31.275374+0000","last_became_active":"2024-03-07T07:20:08.032880+0000","last_became_peered":"2024-03-07T07:20:08.032880+0000","last_unstale":"2024-03-07T07:20:31.275374+0000","last_undegraded":"2024-03-07T07:20:31.275374+0000","last_fullsized":"2024-03-07T07:20:31.275374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.275252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.275252+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.275252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:31.725386+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:23.594241+0000","last_change":"2024-03-07T07:20:23.594281+0000","last_active":"2024-03-07T07:20:23.594241+0000","last_peered":"2024-03-07T07:20:23.594241+0000","last_clean":"2024-03-07T07:20:23.594241+0000","last_became_active":"2024-03-07T07:20:07.024939+0000","last_became_peered":"2024-03-07T07:20:07.024939+0000","last_unstale":"2024-03-07T07:20:23.594241+0000","last_undegraded":"2024-03-07T07:20:23.594241+0000","last_fullsized":"2024-03-07T07:20:23.594241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:23.594133+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:23.594133+0000","last_clean_scrub_stamp":"2024-03-07T07:20:23.594133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:31.121344+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:38.219809+0000","last_change":"2024-03-07T07:20:38.219832+0000","last_active":"2024-03-07T07:20:38.219809+0000","last_peered":"2024-03-07T07:20:38.219809+0000","last_clean":"2024-03-07T07:20:38.219809+0000","last_became_active":"2024-03-07T07:20:08.039907+0000","last_became_peered":"2024-03-07T07:20:08.039907+0000","last_unstale":"2024-03-07T07:20:38.219809+0000","last_undegraded":"2024-03-07T07:20:38.219809+0000","last_fullsized":"2024-03-07T07:20:38.219809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:38.219718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:38.219718+0000","last_clean_scrub_stamp":"2024-03-07T07:20:38.219718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:44:44.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.468303+0000","last_change":"2024-03-07T07:20:27.468352+0000","last_active":"2024-03-07T07:20:27.468303+0000","last_peered":"2024-03-07T07:20:27.468303+0000","last_clean":"2024-03-07T07:20:27.468303+0000","last_became_active":"2024-03-07T07:20:07.032815+0000","last_became_peered":"2024-03-07T07:20:07.032815+0000","last_unstale":"2024-03-07T07:20:27.468303+0000","last_undegraded":"2024-03-07T07:20:27.468303+0000","last_fullsized":"2024-03-07T07:20:27.468303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:27.468188+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:27.468188+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.468188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:06.301869+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.492596+0000","last_change":"2024-03-07T07:20:33.492650+0000","last_active":"2024-03-07T07:20:33.492596+0000","last_peered":"2024-03-07T07:20:33.492596+0000","last_clean":"2024-03-07T07:20:33.492596+0000","last_became_active":"2024-03-07T07:20:08.042946+0000","last_became_peered":"2024-03-07T07:20:08.042946+0000","last_unstale":"2024-03-07T07:20:33.492596+0000","last_undegraded":"2024-03-07T07:20:33.492596+0000","last_fullsized":"2024-03-07T07:20:33.492596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.492487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.492487+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.492487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:13.308386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:23.470674+0000","last_change":"2024-03-07T07:20:23.470722+0000","last_active":"2024-03-07T07:20:23.470674+0000","last_peered":"2024-03-07T07:20:23.470674+0000","last_clean":"2024-03-07T07:20:23.470674+0000","last_became_active":"2024-03-07T07:20:07.028076+0000","last_became_peered":"2024-03-07T07:20:07.028076+0000","last_unstale":"2024-03-07T07:20:23.470674+0000","last_undegraded":"2024-03-07T07:20:23.470674+0000","last_fullsized":"2024-03-07T07:20:23.470674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:23.470554+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:23.470554+0000","last_clean_scrub_stamp":"2024-03-07T07:20:23.470554+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:52:07.876153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.439009+0000","last_change":"2024-03-07T07:20:31.439043+0000","last_active":"2024-03-07T07:20:31.439009+0000","last_peered":"2024-03-07T07:20:31.439009+0000","last_clean":"2024-03-07T07:20:31.439009+0000","last_became_active":"2024-03-07T07:20:08.033575+0000","last_became_peered":"2024-03-07T07:20:08.033575+0000","last_unstale":"2024-03-07T07:20:31.439009+0000","last_undegraded":"2024-03-07T07:20:31.439009+0000","last_fullsized":"2024-03-07T07:20:31.439009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.438907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.438907+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.438907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:34.046498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.248261+0000","last_change":"2024-03-07T07:20:31.248309+0000","last_active":"2024-03-07T07:20:31.248261+0000","last_peered":"2024-03-07T07:20:31.248261+0000","last_clean":"2024-03-07T07:20:31.248261+0000","last_became_active":"2024-03-07T07:20:07.027264+0000","last_became_peered":"2024-03-07T07:20:07.027264+0000","last_unstale":"2024-03-07T07:20:31.248261+0000","last_undegraded":"2024-03-07T07:20:31.248261+0000","last_fullsized":"2024-03-07T07:20:31.248261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:31.248136+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:31.248136+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.248136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:10.388114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.921627+0000","last_change":"2024-03-07T07:20:31.921665+0000","last_active":"2024-03-07T07:20:31.921627+0000","last_peered":"2024-03-07T07:20:31.921627+0000","last_clean":"2024-03-07T07:20:31.921627+0000","last_became_active":"2024-03-07T07:20:08.029776+0000","last_became_peered":"2024-03-07T07:20:08.029776+0000","last_unstale":"2024-03-07T07:20:31.921627+0000","last_undegraded":"2024-03-07T07:20:31.921627+0000","last_fullsized":"2024-03-07T07:20:31.921627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.921523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.921523+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.921523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:28.202647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.279481+0000","last_change":"2024-03-07T07:20:32.279506+0000","last_active":"2024-03-07T07:20:32.279481+0000","last_peered":"2024-03-07T07:20:32.279481+0000","last_clean":"2024-03-07T07:20:32.279481+0000","last_became_active":"2024-03-07T07:20:07.029513+0000","last_became_peered":"2024-03-07T07:20:07.029513+0000","last_unstale":"2024-03-07T07:20:32.279481+0000","last_undegraded":"2024-03-07T07:20:32.279481+0000","last_fullsized":"2024-03-07T07:20:32.279481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:32.279387+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:32.279387+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.279387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:09.214460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.280869+0000","last_change":"2024-03-07T07:20:33.280913+0000","last_active":"2024-03-07T07:20:33.280869+0000","last_peered":"2024-03-07T07:20:33.280869+0000","last_clean":"2024-03-07T07:20:33.280869+0000","last_became_active":"2024-03-07T07:20:08.027771+0000","last_became_peered":"2024-03-07T07:20:08.027771+0000","last_unstale":"2024-03-07T07:20:33.280869+0000","last_undegraded":"2024-03-07T07:20:33.280869+0000","last_fullsized":"2024-03-07T07:20:33.280869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.280770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.280770+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.280770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:09:16.341816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.469330+0000","last_change":"2024-03-07T07:20:29.469350+0000","last_active":"2024-03-07T07:20:29.469330+0000","last_peered":"2024-03-07T07:20:29.469330+0000","last_clean":"2024-03-07T07:20:29.469330+0000","last_became_active":"2024-03-07T07:20:07.033122+0000","last_became_peered":"2024-03-07T07:20:07.033122+0000","last_unstale":"2024-03-07T07:20:29.469330+0000","last_undegraded":"2024-03-07T07:20:29.469330+0000","last_fullsized":"2024-03-07T07:20:29.469330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.469266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.469266+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.469266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:42.129281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.514425+0000","last_change":"2024-03-07T07:20:27.514461+0000","last_active":"2024-03-07T07:20:27.514425+0000","last_peered":"2024-03-07T07:20:27.514425+0000","last_clean":"2024-03-07T07:20:27.514425+0000","last_became_active":"2024-03-07T07:20:08.032377+0000","last_became_peered":"2024-03-07T07:20:08.032377+0000","last_unstale":"2024-03-07T07:20:27.514425+0000","last_undegraded":"2024-03-07T07:20:27.514425+0000","last_fullsized":"2024-03-07T07:20:27.514425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.514313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.514313+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.514313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:04:48.750866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.329249+0000","last_change":"2024-03-07T07:20:29.329276+0000","last_active":"2024-03-07T07:20:29.329249+0000","last_peered":"2024-03-07T07:20:29.329249+0000","last_clean":"2024-03-07T07:20:29.329249+0000","last_became_active":"2024-03-07T07:20:07.026752+0000","last_became_peered":"2024-03-07T07:20:07.026752+0000","last_unstale":"2024-03-07T07:20:29.329249+0000","last_undegraded":"2024-03-07T07:20:29.329249+0000","last_fullsized":"2024-03-07T07:20:29.329249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.329164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.329164+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.329164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:17:29.779298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.958179+0000","last_change":"2024-03-07T07:20:30.958214+0000","last_active":"2024-03-07T07:20:30.958179+0000","last_peered":"2024-03-07T07:20:30.958179+0000","last_clean":"2024-03-07T07:20:30.958179+0000","last_became_active":"2024-03-07T07:20:08.028077+0000","last_became_peered":"2024-03-07T07:20:08.028077+0000","last_unstale":"2024-03-07T07:20:30.958179+0000","last_undegraded":"2024-03-07T07:20:30.958179+0000","last_fullsized":"2024-03-07T07:20:30.958179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.958087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.958087+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.958087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:51:43.618103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.454112+0000","last_change":"2024-03-07T07:20:25.454132+0000","last_active":"2024-03-07T07:20:25.454112+0000","last_peered":"2024-03-07T07:20:25.454112+0000","last_clean":"2024-03-07T07:20:25.454112+0000","last_became_active":"2024-03-07T07:20:07.030140+0000","last_became_peered":"2024-03-07T07:20:07.030140+0000","last_unstale":"2024-03-07T07:20:25.454112+0000","last_undegraded":"2024-03-07T07:20:25.454112+0000","last_fullsized":"2024-03-07T07:20:25.454112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.454040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.454040+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.454040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T19:04:46.690594+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.367621+0000","last_change":"2024-03-07T07:20:35.367640+0000","last_active":"2024-03-07T07:20:35.367621+0000","last_peered":"2024-03-07T07:20:35.367621+0000","last_clean":"2024-03-07T07:20:35.367621+0000","last_became_active":"2024-03-07T07:20:08.034308+0000","last_became_peered":"2024-03-07T07:20:08.034308+0000","last_unstale":"2024-03-07T07:20:35.367621+0000","last_undegraded":"2024-03-07T07:20:35.367621+0000","last_fullsized":"2024-03-07T07:20:35.367621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.367548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.367548+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.367548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:54:24.987586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.536098+0000","last_change":"2024-03-07T07:20:26.536135+0000","last_active":"2024-03-07T07:20:26.536098+0000","last_peered":"2024-03-07T07:20:26.536098+0000","last_clean":"2024-03-07T07:20:26.536098+0000","last_became_active":"2024-03-07T07:20:07.029730+0000","last_became_peered":"2024-03-07T07:20:07.029730+0000","last_unstale":"2024-03-07T07:20:26.536098+0000","last_undegraded":"2024-03-07T07:20:26.536098+0000","last_fullsized":"2024-03-07T07:20:26.536098+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.536002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.536002+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.536002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:53.780258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.446775+0000","last_change":"2024-03-07T07:20:33.446819+0000","last_active":"2024-03-07T07:20:33.446775+0000","last_peered":"2024-03-07T07:20:33.446775+0000","last_clean":"2024-03-07T07:20:33.446775+0000","last_became_active":"2024-03-07T07:20:08.030617+0000","last_became_peered":"2024-03-07T07:20:08.030617+0000","last_unstale":"2024-03-07T07:20:33.446775+0000","last_undegraded":"2024-03-07T07:20:33.446775+0000","last_fullsized":"2024-03-07T07:20:33.446775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.446685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.446685+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.446685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:22:11.407928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.029315+0000","last_change":"2024-03-07T07:20:25.029343+0000","last_active":"2024-03-07T07:20:25.029315+0000","last_peered":"2024-03-07T07:20:25.029315+0000","last_clean":"2024-03-07T07:20:25.029315+0000","last_became_active":"2024-03-07T07:20:07.028215+0000","last_became_peered":"2024-03-07T07:20:07.028215+0000","last_unstale":"2024-03-07T07:20:25.029315+0000","last_undegraded":"2024-03-07T07:20:25.029315+0000","last_fullsized":"2024-03-07T07:20:25.029315+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-03-07T07:20:25.029221+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-03-07T07:20:25.029221+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.029221+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:04.169199+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.799734+0000","last_change":"2024-03-07T07:20:35.799774+0000","last_active":"2024-03-07T07:20:35.799734+0000","last_peered":"2024-03-07T07:20:35.799734+0000","last_clean":"2024-03-07T07:20:35.799734+0000","last_became_active":"2024-03-07T07:20:08.038394+0000","last_became_peered":"2024-03-07T07:20:08.038394+0000","last_unstale":"2024-03-07T07:20:35.799734+0000","last_undegraded":"2024-03-07T07:20:35.799734+0000","last_fullsized":"2024-03-07T07:20:35.799734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.799630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.799630+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.799630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:57:58.471370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'10","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.435236+0000","last_change":"2024-03-07T07:20:30.435261+0000","last_active":"2024-03-07T07:20:30.435236+0000","last_peered":"2024-03-07T07:20:30.435236+0000","last_clean":"2024-03-07T07:20:30.435236+0000","last_became_active":"2024-03-07T07:20:07.035556+0000","last_became_peered":"2024-03-07T07:20:07.035556+0000","last_unstale":"2024-03-07T07:20:30.435236+0000","last_undegraded":"2024-03-07T07:20:30.435236+0000","last_fullsized":"2024-03-07T07:20:30.435236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-03-07T07:20:30.435166+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-03-07T07:20:30.435166+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.435166+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:26.716290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7145,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.557533+0000","last_change":"2024-03-07T07:20:30.557553+0000","last_active":"2024-03-07T07:20:30.557533+0000","last_peered":"2024-03-07T07:20:30.557533+0000","last_clean":"2024-03-07T07:20:30.557533+0000","last_became_active":"2024-03-07T07:20:08.034131+0000","last_became_peered":"2024-03-07T07:20:08.034131+0000","last_unstale":"2024-03-07T07:20:30.557533+0000","last_undegraded":"2024-03-07T07:20:30.557533+0000","last_fullsized":"2024-03-07T07:20:30.557533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.557466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.557466+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.557466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:49.526826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.395187+0000","last_change":"2024-03-07T07:20:27.395238+0000","last_active":"2024-03-07T07:20:27.395187+0000","last_peered":"2024-03-07T07:20:27.395187+0000","last_clean":"2024-03-07T07:20:27.395187+0000","last_became_active":"2024-03-07T07:20:07.025432+0000","last_became_peered":"2024-03-07T07:20:07.025432+0000","last_unstale":"2024-03-07T07:20:27.395187+0000","last_undegraded":"2024-03-07T07:20:27.395187+0000","last_fullsized":"2024-03-07T07:20:27.395187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:27.395067+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:27.395067+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.395067+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:14:54.639323+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:39.266540+0000","last_change":"2024-03-07T07:20:39.266563+0000","last_active":"2024-03-07T07:20:39.266540+0000","last_peered":"2024-03-07T07:20:39.266540+0000","last_clean":"2024-03-07T07:20:39.266540+0000","last_became_active":"2024-03-07T07:20:08.036671+0000","last_became_peered":"2024-03-07T07:20:08.036671+0000","last_unstale":"2024-03-07T07:20:39.266540+0000","last_undegraded":"2024-03-07T07:20:39.266540+0000","last_fullsized":"2024-03-07T07:20:39.266540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:39.266461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:39.266461+0000","last_clean_scrub_stamp":"2024-03-07T07:20:39.266461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:45:13.723890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.601194+0000","last_change":"2024-03-07T07:20:24.601214+0000","last_active":"2024-03-07T07:20:24.601194+0000","last_peered":"2024-03-07T07:20:24.601194+0000","last_clean":"2024-03-07T07:20:24.601194+0000","last_became_active":"2024-03-07T07:20:07.026073+0000","last_became_peered":"2024-03-07T07:20:07.026073+0000","last_unstale":"2024-03-07T07:20:24.601194+0000","last_undegraded":"2024-03-07T07:20:24.601194+0000","last_fullsized":"2024-03-07T07:20:24.601194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.601115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.601115+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.601115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:51.385830+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.484629+0000","last_change":"2024-03-07T07:20:30.484650+0000","last_active":"2024-03-07T07:20:30.484629+0000","last_peered":"2024-03-07T07:20:30.484629+0000","last_clean":"2024-03-07T07:20:30.484629+0000","last_became_active":"2024-03-07T07:20:08.033612+0000","last_became_peered":"2024-03-07T07:20:08.033612+0000","last_unstale":"2024-03-07T07:20:30.484629+0000","last_undegraded":"2024-03-07T07:20:30.484629+0000","last_fullsized":"2024-03-07T07:20:30.484629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.484538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.484538+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.484538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:07:35.617344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.454535+0000","last_change":"2024-03-07T07:20:26.454556+0000","last_active":"2024-03-07T07:20:26.454535+0000","last_peered":"2024-03-07T07:20:26.454535+0000","last_clean":"2024-03-07T07:20:26.454535+0000","last_became_active":"2024-03-07T07:20:07.032344+0000","last_became_peered":"2024-03-07T07:20:07.032344+0000","last_unstale":"2024-03-07T07:20:26.454535+0000","last_undegraded":"2024-03-07T07:20:26.454535+0000","last_fullsized":"2024-03-07T07:20:26.454535+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.454455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.454455+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.454455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:14.787910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.310304+0000","last_change":"2024-03-07T07:20:32.310329+0000","last_active":"2024-03-07T07:20:32.310304+0000","last_peered":"2024-03-07T07:20:32.310304+0000","last_clean":"2024-03-07T07:20:32.310304+0000","last_became_active":"2024-03-07T07:20:08.036124+0000","last_became_peered":"2024-03-07T07:20:08.036124+0000","last_unstale":"2024-03-07T07:20:32.310304+0000","last_undegraded":"2024-03-07T07:20:32.310304+0000","last_fullsized":"2024-03-07T07:20:32.310304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.310206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.310206+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.310206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:59.094921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.468180+0000","last_change":"2024-03-07T07:20:28.468211+0000","last_active":"2024-03-07T07:20:28.468180+0000","last_peered":"2024-03-07T07:20:28.468180+0000","last_clean":"2024-03-07T07:20:28.468180+0000","last_became_active":"2024-03-07T07:20:07.032932+0000","last_became_peered":"2024-03-07T07:20:07.032932+0000","last_unstale":"2024-03-07T07:20:28.468180+0000","last_undegraded":"2024-03-07T07:20:28.468180+0000","last_fullsized":"2024-03-07T07:20:28.468180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'8","last_scrub_stamp":"2024-03-07T07:20:28.468107+0000","last_deep_scrub":"32'8","last_deep_scrub_stamp":"2024-03-07T07:20:28.468107+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.468107+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:47.437467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:35.192563+0000","last_change":"2024-03-07T07:20:35.192591+0000","last_active":"2024-03-07T07:20:35.192563+0000","last_peered":"2024-03-07T07:20:35.192563+0000","last_clean":"2024-03-07T07:20:35.192563+0000","last_became_active":"2024-03-07T07:20:08.036629+0000","last_became_peered":"2024-03-07T07:20:08.036629+0000","last_unstale":"2024-03-07T07:20:35.192563+0000","last_undegraded":"2024-03-07T07:20:35.192563+0000","last_fullsized":"2024-03-07T07:20:35.192563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:35.192461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:35.192461+0000","last_clean_scrub_stamp":"2024-03-07T07:20:35.192461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:31:08.707955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.536215+0000","last_change":"2024-03-07T07:20:26.536258+0000","last_active":"2024-03-07T07:20:26.536215+0000","last_peered":"2024-03-07T07:20:26.536215+0000","last_clean":"2024-03-07T07:20:26.536215+0000","last_became_active":"2024-03-07T07:20:07.032579+0000","last_became_peered":"2024-03-07T07:20:07.032579+0000","last_unstale":"2024-03-07T07:20:26.536215+0000","last_undegraded":"2024-03-07T07:20:26.536215+0000","last_fullsized":"2024-03-07T07:20:26.536215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:26.536134+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:26.536134+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.536134+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:46:10.997014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.848503+0000","last_change":"2024-03-07T07:20:33.848526+0000","last_active":"2024-03-07T07:20:33.848503+0000","last_peered":"2024-03-07T07:20:33.848503+0000","last_clean":"2024-03-07T07:20:33.848503+0000","last_became_active":"2024-03-07T07:20:08.035211+0000","last_became_peered":"2024-03-07T07:20:08.035211+0000","last_unstale":"2024-03-07T07:20:33.848503+0000","last_undegraded":"2024-03-07T07:20:33.848503+0000","last_fullsized":"2024-03-07T07:20:33.848503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.848402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.848402+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.848402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:29:19.906165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.378151+0000","last_change":"2024-03-07T07:20:28.378195+0000","last_active":"2024-03-07T07:20:28.378151+0000","last_peered":"2024-03-07T07:20:28.378151+0000","last_clean":"2024-03-07T07:20:28.378151+0000","last_became_active":"2024-03-07T07:20:07.026258+0000","last_became_peered":"2024-03-07T07:20:07.026258+0000","last_unstale":"2024-03-07T07:20:28.378151+0000","last_undegraded":"2024-03-07T07:20:28.378151+0000","last_fullsized":"2024-03-07T07:20:28.378151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:28.378024+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:28.378024+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.378024+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:48:43.989608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.518868+0000","last_change":"2024-03-07T07:20:29.518923+0000","last_active":"2024-03-07T07:20:29.518868+0000","last_peered":"2024-03-07T07:20:29.518868+0000","last_clean":"2024-03-07T07:20:29.518868+0000","last_became_active":"2024-03-07T07:20:08.033591+0000","last_became_peered":"2024-03-07T07:20:08.033591+0000","last_unstale":"2024-03-07T07:20:29.518868+0000","last_undegraded":"2024-03-07T07:20:29.518868+0000","last_fullsized":"2024-03-07T07:20:29.518868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.518792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.518792+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.518792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:08.352473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.366239+0000","last_change":"2024-03-07T07:20:26.366272+0000","last_active":"2024-03-07T07:20:26.366239+0000","last_peered":"2024-03-07T07:20:26.366239+0000","last_clean":"2024-03-07T07:20:26.366239+0000","last_became_active":"2024-03-07T07:20:07.023387+0000","last_became_peered":"2024-03-07T07:20:07.023387+0000","last_unstale":"2024-03-07T07:20:26.366239+0000","last_undegraded":"2024-03-07T07:20:26.366239+0000","last_fullsized":"2024-03-07T07:20:26.366239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1","last_scrub_stamp":"2024-03-07T07:20:26.366132+0000","last_deep_scrub":"32'1","last_deep_scrub_stamp":"2024-03-07T07:20:26.366132+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.366132+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:11:00.095407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.493071+0000","last_change":"2024-03-07T07:20:26.493090+0000","last_active":"2024-03-07T07:20:26.493071+0000","last_peered":"2024-03-07T07:20:26.493071+0000","last_clean":"2024-03-07T07:20:26.493071+0000","last_became_active":"2024-03-07T07:20:08.030187+0000","last_became_peered":"2024-03-07T07:20:08.030187+0000","last_unstale":"2024-03-07T07:20:26.493071+0000","last_undegraded":"2024-03-07T07:20:26.493071+0000","last_fullsized":"2024-03-07T07:20:26.493071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.493004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.493004+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.493004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:48:14.950271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.375621+0000","last_change":"2024-03-07T07:20:24.375644+0000","last_active":"2024-03-07T07:20:24.375621+0000","last_peered":"2024-03-07T07:20:24.375621+0000","last_clean":"2024-03-07T07:20:24.375621+0000","last_became_active":"2024-03-07T07:20:07.029444+0000","last_became_peered":"2024-03-07T07:20:07.029444+0000","last_unstale":"2024-03-07T07:20:24.375621+0000","last_undegraded":"2024-03-07T07:20:24.375621+0000","last_fullsized":"2024-03-07T07:20:24.375621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:24.375526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:24.375526+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.375526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:54:48.149456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.460383+0000","last_change":"2024-03-07T07:20:28.460407+0000","last_active":"2024-03-07T07:20:28.460383+0000","last_peered":"2024-03-07T07:20:28.460383+0000","last_clean":"2024-03-07T07:20:28.460383+0000","last_became_active":"2024-03-07T07:20:08.031842+0000","last_became_peered":"2024-03-07T07:20:08.031842+0000","last_unstale":"2024-03-07T07:20:28.460383+0000","last_undegraded":"2024-03-07T07:20:28.460383+0000","last_fullsized":"2024-03-07T07:20:28.460383+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.460298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.460298+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.460298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:35.987508+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.579640+0000","last_change":"2024-03-07T07:20:25.579685+0000","last_active":"2024-03-07T07:20:25.579640+0000","last_peered":"2024-03-07T07:20:25.579640+0000","last_clean":"2024-03-07T07:20:25.579640+0000","last_became_active":"2024-03-07T07:20:07.027874+0000","last_became_peered":"2024-03-07T07:20:07.027874+0000","last_unstale":"2024-03-07T07:20:25.579640+0000","last_undegraded":"2024-03-07T07:20:25.579640+0000","last_fullsized":"2024-03-07T07:20:25.579640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:25.579514+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:25.579514+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.579514+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:22:02.747171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:29.411709+0000","last_change":"2024-03-07T07:20:29.411730+0000","last_active":"2024-03-07T07:20:29.411709+0000","last_peered":"2024-03-07T07:20:29.411709+0000","last_clean":"2024-03-07T07:20:29.411709+0000","last_became_active":"2024-03-07T07:20:08.032242+0000","last_became_peered":"2024-03-07T07:20:08.032242+0000","last_unstale":"2024-03-07T07:20:29.411709+0000","last_undegraded":"2024-03-07T07:20:29.411709+0000","last_fullsized":"2024-03-07T07:20:29.411709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:29.411633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:29.411633+0000","last_clean_scrub_stamp":"2024-03-07T07:20:29.411633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:23.085145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.217580+0000","last_change":"2024-03-07T07:20:27.217614+0000","last_active":"2024-03-07T07:20:27.217580+0000","last_peered":"2024-03-07T07:20:27.217580+0000","last_clean":"2024-03-07T07:20:27.217580+0000","last_became_active":"2024-03-07T07:20:07.028174+0000","last_became_peered":"2024-03-07T07:20:07.028174+0000","last_unstale":"2024-03-07T07:20:27.217580+0000","last_undegraded":"2024-03-07T07:20:27.217580+0000","last_fullsized":"2024-03-07T07:20:27.217580+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.217496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.217496+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.217496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:20.891008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.447888+0000","last_change":"2024-03-07T07:20:27.447930+0000","last_active":"2024-03-07T07:20:27.447888+0000","last_peered":"2024-03-07T07:20:27.447888+0000","last_clean":"2024-03-07T07:20:27.447888+0000","last_became_active":"2024-03-07T07:20:08.031021+0000","last_became_peered":"2024-03-07T07:20:08.031021+0000","last_unstale":"2024-03-07T07:20:27.447888+0000","last_undegraded":"2024-03-07T07:20:27.447888+0000","last_fullsized":"2024-03-07T07:20:27.447888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.447800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.447800+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.447800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:12.901646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.295795+0000","last_change":"2024-03-07T07:20:30.295832+0000","last_active":"2024-03-07T07:20:30.295795+0000","last_peered":"2024-03-07T07:20:30.295795+0000","last_clean":"2024-03-07T07:20:30.295795+0000","last_became_active":"2024-03-07T07:20:07.028087+0000","last_became_peered":"2024-03-07T07:20:07.028087+0000","last_unstale":"2024-03-07T07:20:30.295795+0000","last_undegraded":"2024-03-07T07:20:30.295795+0000","last_fullsized":"2024-03-07T07:20:30.295795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.295697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.295697+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.295697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:46:57.080413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.495363+0000","last_change":"2024-03-07T07:20:28.495387+0000","last_active":"2024-03-07T07:20:28.495363+0000","last_peered":"2024-03-07T07:20:28.495363+0000","last_clean":"2024-03-07T07:20:28.495363+0000","last_became_active":"2024-03-07T07:20:08.032765+0000","last_became_peered":"2024-03-07T07:20:08.032765+0000","last_unstale":"2024-03-07T07:20:28.495363+0000","last_undegraded":"2024-03-07T07:20:28.495363+0000","last_fullsized":"2024-03-07T07:20:28.495363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.495254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.495254+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.495254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:03:13.553501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:26.024823+0000","last_change":"2024-03-07T07:20:26.024865+0000","last_active":"2024-03-07T07:20:26.024823+0000","last_peered":"2024-03-07T07:20:26.024823+0000","last_clean":"2024-03-07T07:20:26.024823+0000","last_became_active":"2024-03-07T07:20:07.028967+0000","last_became_peered":"2024-03-07T07:20:07.028967+0000","last_unstale":"2024-03-07T07:20:26.024823+0000","last_undegraded":"2024-03-07T07:20:26.024823+0000","last_fullsized":"2024-03-07T07:20:26.024823+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:26.024712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:26.024712+0000","last_clean_scrub_stamp":"2024-03-07T07:20:26.024712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:02.959784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.483953+0000","last_change":"2024-03-07T07:20:25.483990+0000","last_active":"2024-03-07T07:20:25.483953+0000","last_peered":"2024-03-07T07:20:25.483953+0000","last_clean":"2024-03-07T07:20:25.483953+0000","last_became_active":"2024-03-07T07:20:08.026126+0000","last_became_peered":"2024-03-07T07:20:08.026126+0000","last_unstale":"2024-03-07T07:20:25.483953+0000","last_undegraded":"2024-03-07T07:20:25.483953+0000","last_fullsized":"2024-03-07T07:20:25.483953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.483850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.483850+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.483850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:05:30.756260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.583448+0000","last_change":"2024-03-07T07:20:25.583473+0000","last_active":"2024-03-07T07:20:25.583448+0000","last_peered":"2024-03-07T07:20:25.583448+0000","last_clean":"2024-03-07T07:20:25.583448+0000","last_became_active":"2024-03-07T07:20:07.026870+0000","last_became_peered":"2024-03-07T07:20:07.026870+0000","last_unstale":"2024-03-07T07:20:25.583448+0000","last_undegraded":"2024-03-07T07:20:25.583448+0000","last_fullsized":"2024-03-07T07:20:25.583448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-03-07T07:20:25.583365+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-03-07T07:20:25.583365+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.583365+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:04.723343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:32.875907+0000","last_change":"2024-03-07T07:20:32.875930+0000","last_active":"2024-03-07T07:20:32.875907+0000","last_peered":"2024-03-07T07:20:32.875907+0000","last_clean":"2024-03-07T07:20:32.875907+0000","last_became_active":"2024-03-07T07:20:08.034830+0000","last_became_peered":"2024-03-07T07:20:08.034830+0000","last_unstale":"2024-03-07T07:20:32.875907+0000","last_undegraded":"2024-03-07T07:20:32.875907+0000","last_fullsized":"2024-03-07T07:20:32.875907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:32.875816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:32.875816+0000","last_clean_scrub_stamp":"2024-03-07T07:20:32.875816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:12.691716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'3","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:24.564515+0000","last_change":"2024-03-07T07:20:24.564564+0000","last_active":"2024-03-07T07:20:24.564515+0000","last_peered":"2024-03-07T07:20:24.564515+0000","last_clean":"2024-03-07T07:20:24.564515+0000","last_became_active":"2024-03-07T07:20:07.027277+0000","last_became_peered":"2024-03-07T07:20:07.027277+0000","last_unstale":"2024-03-07T07:20:24.564515+0000","last_undegraded":"2024-03-07T07:20:24.564515+0000","last_fullsized":"2024-03-07T07:20:24.564515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-03-07T07:20:24.564421+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-03-07T07:20:24.564421+0000","last_clean_scrub_stamp":"2024-03-07T07:20:24.564421+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:18.237933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:30.565219+0000","last_change":"2024-03-07T07:20:30.565245+0000","last_active":"2024-03-07T07:20:30.565219+0000","last_peered":"2024-03-07T07:20:30.565219+0000","last_clean":"2024-03-07T07:20:30.565219+0000","last_became_active":"2024-03-07T07:20:08.035991+0000","last_became_peered":"2024-03-07T07:20:08.035991+0000","last_unstale":"2024-03-07T07:20:30.565219+0000","last_undegraded":"2024-03-07T07:20:30.565219+0000","last_fullsized":"2024-03-07T07:20:30.565219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:30.565130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:30.565130+0000","last_clean_scrub_stamp":"2024-03-07T07:20:30.565130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:09.705108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:22.561419+0000","last_change":"2024-03-07T07:20:22.561437+0000","last_active":"2024-03-07T07:20:22.561419+0000","last_peered":"2024-03-07T07:20:22.561419+0000","last_clean":"2024-03-07T07:20:22.561419+0000","last_became_active":"2024-03-07T07:20:07.022765+0000","last_became_peered":"2024-03-07T07:20:07.022765+0000","last_unstale":"2024-03-07T07:20:22.561419+0000","last_undegraded":"2024-03-07T07:20:22.561419+0000","last_fullsized":"2024-03-07T07:20:22.561419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:22.561336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:22.561336+0000","last_clean_scrub_stamp":"2024-03-07T07:20:22.561336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:13:14.464836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:28.219191+0000","last_change":"2024-03-07T07:20:28.219227+0000","last_active":"2024-03-07T07:20:28.219191+0000","last_peered":"2024-03-07T07:20:28.219191+0000","last_clean":"2024-03-07T07:20:28.219191+0000","last_became_active":"2024-03-07T07:20:08.023099+0000","last_became_peered":"2024-03-07T07:20:08.023099+0000","last_unstale":"2024-03-07T07:20:28.219191+0000","last_undegraded":"2024-03-07T07:20:28.219191+0000","last_fullsized":"2024-03-07T07:20:28.219191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:28.219041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:28.219041+0000","last_clean_scrub_stamp":"2024-03-07T07:20:28.219041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:11:01.948316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:22.598033+0000","last_change":"2024-03-07T07:20:22.598054+0000","last_active":"2024-03-07T07:20:22.598033+0000","last_peered":"2024-03-07T07:20:22.598033+0000","last_clean":"2024-03-07T07:20:22.598033+0000","last_became_active":"2024-03-07T07:20:07.023843+0000","last_became_peered":"2024-03-07T07:20:07.023843+0000","last_unstale":"2024-03-07T07:20:22.598033+0000","last_undegraded":"2024-03-07T07:20:22.598033+0000","last_fullsized":"2024-03-07T07:20:22.598033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:22.597947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:22.597947+0000","last_clean_scrub_stamp":"2024-03-07T07:20:22.597947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:54:08.051793+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:27.582226+0000","last_change":"2024-03-07T07:20:27.582267+0000","last_active":"2024-03-07T07:20:27.582226+0000","last_peered":"2024-03-07T07:20:27.582226+0000","last_clean":"2024-03-07T07:20:27.582226+0000","last_became_active":"2024-03-07T07:20:08.029707+0000","last_became_peered":"2024-03-07T07:20:08.029707+0000","last_unstale":"2024-03-07T07:20:27.582226+0000","last_undegraded":"2024-03-07T07:20:27.582226+0000","last_fullsized":"2024-03-07T07:20:27.582226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:27.582148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:27.582148+0000","last_clean_scrub_stamp":"2024-03-07T07:20:27.582148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:13:19.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:23.596312+0000","last_change":"2024-03-07T07:20:23.596350+0000","last_active":"2024-03-07T07:20:23.596312+0000","last_peered":"2024-03-07T07:20:23.596312+0000","last_clean":"2024-03-07T07:20:23.596312+0000","last_became_active":"2024-03-07T07:20:07.025261+0000","last_became_peered":"2024-03-07T07:20:07.025261+0000","last_unstale":"2024-03-07T07:20:23.596312+0000","last_undegraded":"2024-03-07T07:20:23.596312+0000","last_fullsized":"2024-03-07T07:20:23.596312+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:23.596186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:23.596186+0000","last_clean_scrub_stamp":"2024-03-07T07:20:23.596186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:04:59.634407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:36.360741+0000","last_change":"2024-03-07T07:20:36.360759+0000","last_active":"2024-03-07T07:20:36.360741+0000","last_peered":"2024-03-07T07:20:36.360741+0000","last_clean":"2024-03-07T07:20:36.360741+0000","last_became_active":"2024-03-07T07:20:08.034998+0000","last_became_peered":"2024-03-07T07:20:08.034998+0000","last_unstale":"2024-03-07T07:20:36.360741+0000","last_undegraded":"2024-03-07T07:20:36.360741+0000","last_fullsized":"2024-03-07T07:20:36.360741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:36.360672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:36.360672+0000","last_clean_scrub_stamp":"2024-03-07T07:20:36.360672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:31:37.413085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.355004+0000","last_change":"2024-03-07T07:20:25.355025+0000","last_active":"2024-03-07T07:20:25.355004+0000","last_peered":"2024-03-07T07:20:25.355004+0000","last_clean":"2024-03-07T07:20:25.355004+0000","last_became_active":"2024-03-07T07:20:07.023101+0000","last_became_peered":"2024-03-07T07:20:07.023101+0000","last_unstale":"2024-03-07T07:20:25.355004+0000","last_undegraded":"2024-03-07T07:20:25.355004+0000","last_fullsized":"2024-03-07T07:20:25.355004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.354911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.354911+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.354911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:46:19.028423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:37.170992+0000","last_change":"2024-03-07T07:20:37.171042+0000","last_active":"2024-03-07T07:20:37.170992+0000","last_peered":"2024-03-07T07:20:37.170992+0000","last_clean":"2024-03-07T07:20:37.170992+0000","last_became_active":"2024-03-07T07:20:08.039792+0000","last_became_peered":"2024-03-07T07:20:08.039792+0000","last_unstale":"2024-03-07T07:20:37.170992+0000","last_undegraded":"2024-03-07T07:20:37.170992+0000","last_fullsized":"2024-03-07T07:20:37.170992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:37.170881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:37.170881+0000","last_clean_scrub_stamp":"2024-03-07T07:20:37.170881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:05:39.778472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:31.419580+0000","last_change":"2024-03-07T07:20:31.419619+0000","last_active":"2024-03-07T07:20:31.419580+0000","last_peered":"2024-03-07T07:20:31.419580+0000","last_clean":"2024-03-07T07:20:31.419580+0000","last_became_active":"2024-03-07T07:20:07.036585+0000","last_became_peered":"2024-03-07T07:20:07.036585+0000","last_unstale":"2024-03-07T07:20:31.419580+0000","last_undegraded":"2024-03-07T07:20:31.419580+0000","last_fullsized":"2024-03-07T07:20:31.419580+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:31.419472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:31.419472+0000","last_clean_scrub_stamp":"2024-03-07T07:20:31.419472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:11.235371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:33.433685+0000","last_change":"2024-03-07T07:20:33.433711+0000","last_active":"2024-03-07T07:20:33.433685+0000","last_peered":"2024-03-07T07:20:33.433685+0000","last_clean":"2024-03-07T07:20:33.433685+0000","last_became_active":"2024-03-07T07:20:08.034652+0000","last_became_peered":"2024-03-07T07:20:08.034652+0000","last_unstale":"2024-03-07T07:20:33.433685+0000","last_undegraded":"2024-03-07T07:20:33.433685+0000","last_fullsized":"2024-03-07T07:20:33.433685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:33.433555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:33.433555+0000","last_clean_scrub_stamp":"2024-03-07T07:20:33.433555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:35:13.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.263700+0000","last_change":"2024-03-07T07:20:25.263739+0000","last_active":"2024-03-07T07:20:25.263700+0000","last_peered":"2024-03-07T07:20:25.263700+0000","last_clean":"2024-03-07T07:20:25.263700+0000","last_became_active":"2024-03-07T07:20:07.023209+0000","last_became_peered":"2024-03-07T07:20:07.023209+0000","last_unstale":"2024-03-07T07:20:25.263700+0000","last_undegraded":"2024-03-07T07:20:25.263700+0000","last_fullsized":"2024-03-07T07:20:25.263700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T07:20:25.263585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T07:20:25.263585+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.263585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:47:33.629765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"33'119","reported_seq":204,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-07T07:20:25.434174+0000","last_change":"2024-03-07T07:20:25.434236+0000","last_active":"2024-03-07T07:20:25.434174+0000","last_peered":"2024-03-07T07:20:25.434174+0000","last_clean":"2024-03-07T07:20:25.434174+0000","last_became_active":"2024-03-07T07:19:05.374387+0000","last_became_peered":"2024-03-07T07:19:05.374387+0000","last_unstale":"2024-03-07T07:20:25.434174+0000","last_undegraded":"2024-03-07T07:20:25.434174+0000","last_fullsized":"2024-03-07T07:20:25.434174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2024-03-07T07:20:25.434050+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2024-03-07T07:20:25.434050+0000","last_clean_scrub_stamp":"2024-03-07T07:20:25.434050+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:01:04.574028+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":7943,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":15886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":970752,"data_stored":737074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":147232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":146720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":376832,"data_stored":146686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":393216,"data_stored":153783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":736984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T07:21:06.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-07T07:21:06.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.471+0000 7ff33a55d640 1 Processor -- start 2024-03-07T07:21:06.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.471+0000 7ff33a55d640 1 -- start start 2024-03-07T07:21:06.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.471+0000 7ff33a55d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3341625c0 0x7ff334071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.471+0000 7ff33a55d640 1 -- --> v1:172.21.15.64:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff334071ff0 con 0x7ff3340b7200 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.471+0000 7ff33a55d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3340d23b0 con 0x7ff33415e350 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.471+0000 7ff33a55d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340d2160 con 0x7ff3341625c0 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff338afa640 1 --1- >> v1:172.21.15.64:6789/0 conn(0x7ff3340b7200 0x7ff3340717c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.64:6789/0 says I am v1:172.21.15.64:50562/0 (socket says 172.21.15.64:50562) 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff338afa640 1 -- 172.21.15.64:0/4235680069 learned_addr learned my addr 172.21.15.64:0/4235680069 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff3337fe640 1 --2- 172.21.15.64:0/4235680069 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3341625c0 0x7ff334071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 <== mon.0 v1:172.21.15.64:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910771781 0 0) 0x7ff334071ff0 con 0x7ff3340b7200 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 --> v1:172.21.15.64:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff31c003520 con 0x7ff3340b7200 2024-03-07T07:21:06.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587535440 0 0) 0x7ff3340d23b0 con 0x7ff33415e350 2024-03-07T07:21:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff334071ff0 con 0x7ff33415e350 2024-03-07T07:21:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 <== mon.0 v1:172.21.15.64:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2972168740 0 0) 0x7ff31c003520 con 0x7ff3340b7200 2024-03-07T07:21:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff3337fe640 1 -- 172.21.15.64:0/4235680069 >> v1:172.21.15.120:6789/0 conn(0x7ff33415e350 legacy=0x7ff334161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff3337fe640 1 -- 172.21.15.64:0/4235680069 >> v1:172.21.15.64:6789/0 conn(0x7ff3340b7200 legacy=0x7ff3340717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.472+0000 7ff3337fe640 1 -- 172.21.15.64:0/4235680069 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340d2690 con 0x7ff3341625c0 2024-03-07T07:21:06.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.473+0000 7ff3337fe640 1 --2- 172.21.15.64:0/4235680069 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3341625c0 0x7ff334071280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3280097d0 tx=0x7ff32802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e3c95adde1d079e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.473+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328037070 con 0x7ff3341625c0 2024-03-07T07:21:06.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.473+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff328030510 con 0x7ff3341625c0 2024-03-07T07:21:06.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.473+0000 7ff332ffd640 1 -- 172.21.15.64:0/4235680069 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0019f0 con 0x7ff3341625c0 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.474+0000 7ff33a55d640 1 -- 172.21.15.64:0/4235680069 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3341625c0 msgr2=0x7ff334071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.474+0000 7ff33a55d640 1 --2- 172.21.15.64:0/4235680069 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3341625c0 0x7ff334071280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3280097d0 tx=0x7ff32802b800 comp rx=0 tx=0).stop 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.474+0000 7ff33a55d640 1 -- 172.21.15.64:0/4235680069 shutdown_connections 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.474+0000 7ff33a55d640 1 --2- 172.21.15.64:0/4235680069 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3341625c0 0x7ff334071280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.474+0000 7ff33a55d640 1 -- 172.21.15.64:0/4235680069 >> 172.21.15.64:0/4235680069 conn(0x7ff3340d0e70 msgr2=0x7ff3340d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.474+0000 7ff33a55d640 1 -- 172.21.15.64:0/4235680069 shutdown_connections 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.474+0000 7ff33a55d640 1 -- 172.21.15.64:0/4235680069 wait complete. 2024-03-07T07:21:06.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.475+0000 7ff33a55d640 1 Processor -- start 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.475+0000 7ff33a55d640 1 -- start start 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff33a55d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3340b7200 0x7ff334145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff33a55d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33415e350 0x7ff3340c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff33a55d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3341625c0 0x7ff334155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff33a55d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341691e0 con 0x7ff33415e350 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff33a55d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33415e730 con 0x7ff3341625c0 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff33a55d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff334169360 con 0x7ff3340b7200 2024-03-07T07:21:06.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33415e350 0x7ff3340c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33415e350 0x7ff3340c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33136/0 (socket says 172.21.15.64:33136) 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 -- 172.21.15.64:0/2563119755 learned_addr learned my addr 172.21.15.64:0/2563119755 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff338afa640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3341625c0 0x7ff334155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff333fff640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3340b7200 0x7ff334145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 -- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3340b7200 msgr2=0x7ff334145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3340b7200 0x7ff334145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 -- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3341625c0 msgr2=0x7ff334155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3341625c0 0x7ff334155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 -- 172.21.15.64:0/2563119755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340c7660 con 0x7ff33415e350 2024-03-07T07:21:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.476+0000 7ff3337fe640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33415e350 0x7ff3340c9630 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff3280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.477+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328041020 con 0x7ff33415e350 2024-03-07T07:21:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.477+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32803c870 con 0x7ff33415e350 2024-03-07T07:21:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.477+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3340c7940 con 0x7ff33415e350 2024-03-07T07:21:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.477+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328037040 con 0x7ff33415e350 2024-03-07T07:21:06.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.477+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3340c7f00 con 0x7ff33415e350 2024-03-07T07:21:06.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.478+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff334152fe0 con 0x7ff33415e350 2024-03-07T07:21:06.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.479+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff328044020 con 0x7ff33415e350 2024-03-07T07:21:06.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.479+0000 7ff330ff9640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.479+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3280abb30 con 0x7ff33415e350 2024-03-07T07:21:06.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.479+0000 7ff333fff640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.480+0000 7ff333fff640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff32006b4d0 0x7ff32006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff3240011e0 tx=0x7ff324009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.484+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff32804b030 con 0x7ff33415e350 2024-03-07T07:21:06.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.663+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ff3340cfe00 con 0x7ff33415e350 2024-03-07T07:21:06.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.666+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v11)=0 v11) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff3340cfe00 con 0x7ff33415e350 2024-03-07T07:21:06.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.667+0000 7ff330ff9640 1 -- 172.21.15.64:0/2563119755 <== mon.0 v2:172.21.15.64:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff32807a360 con 0x7ff33415e350 2024-03-07T07:21:06.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff32006b4d0 msgr2=0x7ff32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff32006b4d0 0x7ff32006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff3240011e0 tx=0x7ff324009040 comp rx=0 tx=0).stop 2024-03-07T07:21:06.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33415e350 msgr2=0x7ff3340c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33415e350 0x7ff3340c9630 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff3280046c0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 shutdown_connections 2024-03-07T07:21:06.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff33415e350 0x7ff3340c9630 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3341625c0 0x7ff334155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 --2- 172.21.15.64:0/2563119755 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3340b7200 0x7ff334145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.668+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 >> 172.21.15.64:0/2563119755 conn(0x7ff3340d0e70 msgr2=0x7ff3340b9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:21:06.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.669+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 shutdown_connections 2024-03-07T07:21:06.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.669+0000 7ff33a55d640 1 -- 172.21.15.64:0/2563119755 wait complete. 2024-03-07T07:21:06.680 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-07T07:21:06.680 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T07:21:06.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.781+0000 7f74f8c1a640 1 Processor -- start 2024-03-07T07:21:06.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.781+0000 7f74f8c1a640 1 -- start start 2024-03-07T07:21:06.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.782+0000 7f74f8c1a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f74f4145bf0 0x7f74f40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.782+0000 7f74f8c1a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40d1860 0x7f74f406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.782+0000 7f74f8c1a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f40d0aa0 con 0x7f74f40b7390 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.782+0000 7f74f8c1a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f40d13d0 con 0x7f74f4145bf0 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.782+0000 7f74f8c1a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40d0d80 con 0x7f74f40d1860 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74e9d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40d1860 0x7f74f406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f259c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f74f4145bf0 0x7f74f40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f2d9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f74f40b7390 0x7f74f415fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.64:38380/0 (socket says 172.21.15.64:38380) 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74e9d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40d1860 0x7f74f406b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.64:60992/0 (socket says 172.21.15.64:60992) 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f259c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f74f4145bf0 0x7f74f40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.64:35544/0 (socket says 172.21.15.64:35544) 2024-03-07T07:21:06.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f2d9d640 1 -- 172.21.15.64:0/2026003547 learned_addr learned my addr 172.21.15.64:0/2026003547 (peer_addr_for_me v1:172.21.15.64:0/0) 2024-03-07T07:21:06.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f259c640 1 -- 172.21.15.64:0/2026003547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40d1860 msgr2=0x7f74f406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T07:21:06.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f259c640 1 --2- 172.21.15.64:0/2026003547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40d1860 0x7f74f406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f259c640 1 -- 172.21.15.64:0/2026003547 >> v1:172.21.15.120:6789/0 conn(0x7f74f40b7390 legacy=0x7f74f415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f1d9b640 1 -- 172.21.15.64:0/2026003547 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244059565 0 0) 0x7f74f40d0aa0 con 0x7f74f40b7390 2024-03-07T07:21:06.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.783+0000 7f74f259c640 1 -- 172.21.15.64:0/2026003547 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f40d1060 con 0x7f74f4145bf0 2024-03-07T07:21:06.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.784+0000 7f74f259c640 1 --2- 172.21.15.64:0/2026003547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f4145bf0 0x7f74f40cea70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74f40d1550 tx=0x7f74e402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24bc36886e5045f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.784+0000 7f74f1d9b640 1 -- 172.21.15.64:0/2026003547 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4035020 con 0x7f74f4145bf0 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.784+0000 7f74f1d9b640 1 -- 172.21.15.64:0/2026003547 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f74e4031800 con 0x7f74f4145bf0 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.784+0000 7f74f1d9b640 1 -- 172.21.15.64:0/2026003547 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4031bf0 con 0x7f74f4145bf0 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 -- 172.21.15.64:0/2026003547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f4145bf0 msgr2=0x7f74f40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/2026003547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f4145bf0 0x7f74f40cea70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74f40d1550 tx=0x7f74e402b8a0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 -- 172.21.15.64:0/2026003547 shutdown_connections 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/2026003547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40d1860 0x7f74f406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/2026003547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f4145bf0 0x7f74f40cea70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 -- 172.21.15.64:0/2026003547 >> 172.21.15.64:0/2026003547 conn(0x7f74f406d4c0 msgr2=0x7f74f4066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 -- 172.21.15.64:0/2026003547 shutdown_connections 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.785+0000 7f74f8c1a640 1 -- 172.21.15.64:0/2026003547 wait complete. 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.786+0000 7f74f8c1a640 1 Processor -- start 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.786+0000 7f74f8c1a640 1 -- start start 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f8c1a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40b7390 0x7f74f4071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f8c1a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f40d1860 0x7f74f415f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f8c1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74f4145bf0 0x7f74f41f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f8c1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40cf130 con 0x7f74f4145bf0 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f8c1a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40cefb0 con 0x7f74f40d1860 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f8c1a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40d06c0 con 0x7f74f40b7390 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74f4145bf0 0x7f74f41f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74f4145bf0 0x7f74f41f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33140/0 (socket says 172.21.15.64:33140) 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 -- 172.21.15.64:0/1410706644 learned_addr learned my addr 172.21.15.64:0/1410706644 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f259c640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40b7390 0x7f74f4071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74e9d9b640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f40d1860 0x7f74f415f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 -- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40b7390 msgr2=0x7f74f4071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40b7390 0x7f74f4071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 -- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f40d1860 msgr2=0x7f74f415f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f40d1860 0x7f74f415f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74f2d9d640 1 -- 172.21.15.64:0/1410706644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f415f6a0 con 0x7f74f4145bf0 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.787+0000 7f74e9d9b640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f40d1860 0x7f74f415f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.788+0000 7f74f2d9d640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74f4145bf0 0x7f74f41f3ca0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c7a0 tx=0x7f74ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.788+0000 7f74eb7fe640 1 -- 172.21.15.64:0/1410706644 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec018070 con 0x7f74f4145bf0 2024-03-07T07:21:06.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.788+0000 7f74eb7fe640 1 -- 172.21.15.64:0/1410706644 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f74ec00a640 con 0x7f74f4145bf0 2024-03-07T07:21:06.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.788+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f41f41e0 con 0x7f74f4145bf0 2024-03-07T07:21:06.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.788+0000 7f74eb7fe640 1 -- 172.21.15.64:0/1410706644 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00aad0 con 0x7f74f4145bf0 2024-03-07T07:21:06.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.788+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f4071e10 con 0x7f74f4145bf0 2024-03-07T07:21:06.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.789+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4062c90 con 0x7f74f4145bf0 2024-03-07T07:21:06.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.789+0000 7f74eb7fe640 1 -- 172.21.15.64:0/1410706644 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f74ec01b070 con 0x7f74f4145bf0 2024-03-07T07:21:06.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.790+0000 7f74eb7fe640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74d406b500 0x7f74d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T07:21:06.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.790+0000 7f74eb7fe640 1 -- 172.21.15.64:0/1410706644 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74ec0897b0 con 0x7f74f4145bf0 2024-03-07T07:21:06.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.790+0000 7f74f259c640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74d406b500 0x7f74d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T07:21:06.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.790+0000 7f74f259c640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74d406b500 0x7f74d406d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74e4036000 tx=0x7f74e4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T07:21:06.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:06.798+0000 7f74eb7fe640 1 -- 172.21.15.64:0/1410706644 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f74ec058630 con 0x7f74f4145bf0 2024-03-07T07:21:07.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.018+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f4072860 con 0x7f74f4145bf0 2024-03-07T07:21:07.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.018+0000 7f74eb7fe640 1 -- 172.21.15.64:0/1410706644 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f74ec057f30 con 0x7f74f4145bf0 2024-03-07T07:21:07.020 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-07T07:21:07.020 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6836/1229777071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":1229777071},{"type":"v1","addr":"172.21.15.120:6836","nonce":1229777071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/3785220969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6835","nonce":3785220969},{"type":"v1","addr":"172.21.15.120:6837","nonce":3785220969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5079,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6834/229489259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":229489259},{"type":"v1","addr":"172.21.15.64:6834","nonce":229489259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T07:20:07.375701+0000","modified":"2024-03-07T07:20:09.484751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6835/3131353555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6833","nonce":3131353555},{"type":"v1","addr":"172.21.15.64:6835","nonce":3131353555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5082,"qdb_cluster":[5082]},"id":2}]} 2024-03-07T07:21:07.020 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 14 2024-03-07T07:21:07.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74d406b500 msgr2=0x7f74d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:07.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74d406b500 0x7f74d406d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74e4036000 tx=0x7f74e4008030 comp rx=0 tx=0).stop 2024-03-07T07:21:07.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74f4145bf0 msgr2=0x7f74f41f3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T07:21:07.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74f4145bf0 0x7f74f41f3ca0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c7a0 tx=0x7f74ec00cc60 comp rx=0 tx=0).stop 2024-03-07T07:21:07.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 shutdown_connections 2024-03-07T07:21:07.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:6832/182409167,v1:172.21.15.120:6833/182409167] conn(0x7f74d406b500 0x7f74d406d9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:07.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74f4145bf0 0x7f74f41f3ca0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:07.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f74f40d1860 0x7f74f415f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:07.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 --2- 172.21.15.64:0/1410706644 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f74f40b7390 0x7f74f4071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T07:21:07.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 >> 172.21.15.64:0/1410706644 conn(0x7f74f406d4c0 msgr2=0x7f74f41603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T07:21:07.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 shutdown_connections 2024-03-07T07:21:07.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-03-07T07:21:07.020+0000 7f74f8c1a640 1 -- 172.21.15.64:0/1410706644 wait complete. 2024-03-07T07:21:07.035 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-07T07:21:07.036 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-07T07:21:07.036 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-07T07:21:07.036 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.045 INFO:tasks.ceph.mds.a:Stopped 2024-03-07T07:21:07.045 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-07T07:21:07.046 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.097 INFO:tasks.ceph.mds.c:Stopped 2024-03-07T07:21:07.097 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-07T07:21:07.097 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.109 INFO:tasks.ceph.mds.b:Stopped 2024-03-07T07:21:07.109 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-07T07:21:07.109 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.160 INFO:tasks.ceph.mds.d:Stopped 2024-03-07T07:21:07.161 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-07T07:21:07.161 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-07T07:21:07.161 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.188 INFO:tasks.ceph.osd.0:Stopped 2024-03-07T07:21:07.189 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-07T07:21:07.189 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.255 INFO:tasks.ceph.osd.1:Stopped 2024-03-07T07:21:07.255 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-07T07:21:07.255 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.328 INFO:tasks.ceph.osd.2:Stopped 2024-03-07T07:21:07.329 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-07T07:21:07.329 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.394 INFO:tasks.ceph.osd.3:Stopped 2024-03-07T07:21:07.395 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-07T07:21:07.395 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.425 INFO:tasks.ceph.osd.4:Stopped 2024-03-07T07:21:07.425 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-07T07:21:07.425 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.494 INFO:tasks.ceph.osd.5:Stopped 2024-03-07T07:21:07.495 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-07T07:21:07.495 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.561 INFO:tasks.ceph.osd.6:Stopped 2024-03-07T07:21:07.562 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-07T07:21:07.562 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.633 INFO:tasks.ceph.osd.7:Stopped 2024-03-07T07:21:07.633 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-07T07:21:07.633 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-07T07:21:07.633 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.658 INFO:tasks.ceph.mgr.y:Stopped 2024-03-07T07:21:07.658 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-07T07:21:07.658 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.702 INFO:tasks.ceph.mgr.x:Stopped 2024-03-07T07:21:07.703 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-07T07:21:07.703 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-07T07:21:07.703 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.714 INFO:tasks.ceph.mon.a:Stopped 2024-03-07T07:21:07.714 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-07T07:21:07.714 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.755 INFO:tasks.ceph.mon.b:Stopped 2024-03-07T07:21:07.756 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-07T07:21:07.756 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T07:21:07.808 INFO:tasks.ceph.mon.c:Stopped 2024-03-07T07:21:07.808 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-07T07:21:07.809 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-07T07:21:07.842 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:21:07.842 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-07T07:21:08.070 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:21:08.070 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-07T07:21:08.143 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:21:08.143 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-07T07:21:08.202 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:21:08.202 DEBUG:teuthology.orchestra.run.smithi064:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-07T07:21:08.251 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:21:08.251 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-07T07:21:08.406 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:21:08.406 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-07T07:21:08.453 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:21:08.454 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-07T07:21:08.503 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:21:08.503 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-07T07:21:08.553 INFO:tasks.ceph:Archiving mon data... 2024-03-07T07:21:08.554 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/data/mon.a.tgz 2024-03-07T07:21:08.555 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-03-07T07:21:08.569 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.RA6dj6aFPV 2024-03-07T07:21:08.570 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RA6dj6aFPV 2024-03-07T07:21:08.701 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.RA6dj6aFPV 2024-03-07T07:21:08.787 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.RA6dj6aFPV is 278KB 2024-03-07T07:21:08.882 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.RA6dj6aFPV 2024-03-07T07:21:08.897 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/data/mon.b.tgz 2024-03-07T07:21:08.897 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-03-07T07:21:08.913 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.Eby6swNn18 2024-03-07T07:21:08.913 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Eby6swNn18 2024-03-07T07:21:09.061 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.Eby6swNn18 2024-03-07T07:21:09.144 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.Eby6swNn18 is 375KB 2024-03-07T07:21:09.239 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.Eby6swNn18 2024-03-07T07:21:09.253 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/data/mon.c.tgz 2024-03-07T07:21:09.253 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-03-07T07:21:09.307 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.D6Mf2PUGg0 2024-03-07T07:21:09.308 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.D6Mf2PUGg0 2024-03-07T07:21:09.441 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.D6Mf2PUGg0 2024-03-07T07:21:09.524 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.D6Mf2PUGg0 is 375KB 2024-03-07T07:21:09.622 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.D6Mf2PUGg0 2024-03-07T07:21:09.637 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-07T07:21:09.637 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-07T07:21:09.640 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-07T07:21:09.723 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-07T07:21:09.758 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-07T07:21:09.759 INFO:tasks.ceph:Archiving crash dumps... 2024-03-07T07:21:09.760 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/remote/smithi064/crash 2024-03-07T07:21:09.761 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-07T07:21:09.793 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/remote/smithi120/crash 2024-03-07T07:21:09.794 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-07T07:21:09.825 INFO:tasks.ceph:Shutting down logrotate 2024-03-07T07:21:09.825 DEBUG:teuthology.orchestra.run.smithi064:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:21:09.837 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T07:21:09.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-07T07:21:09.935 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-07T07:21:09.973 INFO:tasks.ceph:Compressing logs... 2024-03-07T07:21:09.973 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-07T07:21:09.977 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-07T07:21:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-07T07:21:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43493.log 2024-03-07T07:21:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-07T07:21:10.060 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.1.43493.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-07T07:21:10.060 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43493.log.gz 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37545.log 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37585.log 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.37545.log: /var/log/ceph/ceph-osd.4.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37545.log.gz 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37505.log 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43882.log 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.37585.log: /var/log/ceph/ceph-client.admin.37505.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37585.log.gz 2024-03-07T07:21:10.061 INFO:teuthology.orchestra.run.smithi120.stderr: --verbose 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr: -- /var/log/ceph/ceph-osd.7.log 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.3.43882.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37505.log.gz 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43882.log.gz 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37425.log 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37305.log 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.37425.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr: 89.0%/var/log/ceph/ceph-client.admin.37305.log: -- replaced with /var/log/ceph/ceph-client.admin.37425.log.gz 2024-03-07T07:21:10.062 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37305.log.gz 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.37465.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37345.log 2024-03-07T07:21:10.063 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mds.d.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-07T07:21:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-07T07:21:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.37345.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37345.log.gz 2024-03-07T07:21:10.064 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-03-07T07:21:10.073 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.37385.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-03-07T07:21:10.093 INFO:teuthology.orchestra.run.smithi120.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-07T07:21:10.094 INFO:teuthology.orchestra.run.smithi120.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-07T07:21:10.094 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51857.log 2024-03-07T07:21:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 2024-03-07T07:21:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51857.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-07T07:21:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51563.log: gzip 87.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.51857.log.gz -- 2024-03-07T07:21:10.095 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.43399.log 2024-03-07T07:21:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mon.a.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2024-03-07T07:21:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-03-07T07:21:10.095 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51815.log 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43399.log.gz 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37580.log 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51815.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51312.log 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.37580.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51815.log.gz 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31136.log 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37580.log.gz 2024-03-07T07:21:10.096 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43607.log 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.tmp-client.admin.31136.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51312.log.gz 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45351.log 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31136.log.gz 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43607.log.gz 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49674.log 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48756.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45351.log.gz 2024-03-07T07:21:10.097 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48006.log 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49008.log 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49674.log.gz 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47698.log 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49008.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48006.log.gz 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49008.log.gz 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46857.log 2024-03-07T07:21:10.098 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52515.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47698.log.gz 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47487.log 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8%/var/log/ceph/ceph-client.admin.46857.log: -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43480.log 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46857.log.gz 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43480.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gzgzip 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44049.log 2024-03-07T07:21:10.099 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48966.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43480.log.gz 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51773.log 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52151.log 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47193.log 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44049.log.gz 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52151.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51773.log.gz 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43565.log 2024-03-07T07:21:10.100 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52151.log.gz 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50322.log 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50276.log 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47193.log.gz 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50322.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43565.log.gz 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46427.log 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50322.log.gz 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49344.log 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.46427.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.50276.log.gz 2024-03-07T07:21:10.101 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47852.log 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49344.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46427.log.gz 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46941.log 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49344.log.gz 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46899.log 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.46941.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47852.log.gz 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.46899.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46941.log.gz 2024-03-07T07:21:10.102 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50574.log 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46899.log.gz 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50490.log 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51731.log 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50490.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50574.log.gz 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49218.log 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50490.log.gz 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49470.log 2024-03-07T07:21:10.103 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49885.log 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51731.log.gz 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49470.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49218.log.gz 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49470.log.gz 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47403.log 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51186.log 88.0% 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49885.log.gz 2024-03-07T07:21:10.104 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47403.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43923.log 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47403.log.gz 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51605.log 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45225.log 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51186.log.gz 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51605.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43923.log.gz 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45729.log 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51605.log.gz 2024-03-07T07:21:10.105 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43691.log 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45729.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45225.log.gz 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45603.log 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45729.log.gz 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52277.log 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43691.log.gz 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37487.log 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48090.log 2024-03-07T07:21:10.106 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45603.log.gz 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.37487.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52277.log.gz 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51983.log 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37487.log.gz 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44175.log 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48090.log.gz 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44175.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51983.log.gz 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52403.log 2024-03-07T07:21:10.107 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44175.log.gz 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51144.log 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45435.log 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49260.log 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52403.log.gz 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51144.log.gz 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44259.log 2024-03-07T07:21:10.108 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45435.log.gz 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44985.log 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49260.log.gz 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49176.log 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44644.log 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44259.log.gz 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49176.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44985.log.gz 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52475.log 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49176.log.gz 2024-03-07T07:21:10.109 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52319.log 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52475.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44644.log.gz 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44983.log 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52319.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52475.log.gz 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47235.log 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52319.log.gz 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44983.log.gz 2024-03-07T07:21:10.110 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47741.log 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49092.log 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47235.log.gz 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50234.log 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50097.log 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49092.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47741.log.gz 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50234.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49092.log.gz 2024-03-07T07:21:10.111 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52025.log 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50192.log 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50234.log.gz 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50097.log: /var/log/ceph/ceph-client.admin.52025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50682.log 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50097.log.gz 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52025.log.gz 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50192.log.gz 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52193.log 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-03-07T07:21:10.112 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6%/var/log/ceph/ceph-client.admin.52193.log: -- replaced with /var/log/ceph/ceph-client.admin.50682.log.gz 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48132.log 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51228.log 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52193.log.gz 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48132.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45813.log 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48132.log.gz 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44647.log 2024-03-07T07:21:10.113 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51228.log.gz 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49843.log 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44301.log 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49843.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44647.log.gz 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45813.log.gz 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43649.log 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49843.log.gz 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50448.log 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44301.log.gz 2024-03-07T07:21:10.114 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51521.log 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50141.log 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43649.log.gz 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51521.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50448.log.gz 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43522.log 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51521.log.gz 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48636.log 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43522.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50141.log.gz 2024-03-07T07:21:10.115 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51941.log 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43522.log.gz/var/log/ceph/ceph-client.admin.48636.log: 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44640.log 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48636.log.gz 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44091.log 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51941.log.gz 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47109.log 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43777.log 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44640.log.gz 2024-03-07T07:21:10.116 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47109.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44091.log.gz 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45183.log 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47109.log.gz 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50869.log 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43777.log.gz 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46727.log 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.50869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50869.log.gz 2024-03-07T07:21:10.117 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45183.log.gz 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.2.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49759.log 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46727.log.gz 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44648.log 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45561.log 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49759.log.gz 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44648.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44343.log 2024-03-07T07:21:10.118 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44648.log.gz 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48294.log 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44343.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45561.log.gz 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47361.log 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44343.log.gz 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48294.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48294.log.gz 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44971.log 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.1.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47361.log.gz 2024-03-07T07:21:10.119 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49927.log 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49386.log 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49927.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44971.log.gz 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48714.log 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49386.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49927.log.gz 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49632.log 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49386.log.gz 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47964.log 2024-03-07T07:21:10.120 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48714.log: /var/log/ceph/ceph-client.admin.49632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49551.log 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48714.log.gz 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47964.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49632.log.gz 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51899.log 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45477.log 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7%/var/log/ceph/ceph-client.admin.49551.log: -- replaced with /var/log/ceph/ceph-client.admin.47964.log.gz 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51899.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47025.log 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51899.log.gz 2024-03-07T07:21:10.121 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50406.log 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45477.log: /var/log/ceph/ceph-client.admin.47025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51689.log 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45477.log.gz 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47025.log.gz 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48552.log 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50406.log.gz 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43303.log 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44650.log 2024-03-07T07:21:10.122 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48552.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51689.log.gz 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43303.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.48552.log.gz 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47655.log 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43303.log.gz 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44650.log: /var/log/ceph/ceph-client.admin.47655.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44650.log.gz 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50532.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47655.log.gz 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48594.log 2024-03-07T07:21:10.123 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44643.log 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37208.log 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48594.log: /var/log/ceph/ceph-client.admin.44643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49134.log 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48594.log.gz 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44643.log.gz 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.37208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44469.log 2024-03-07T07:21:10.124 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49134.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37208.log.gz 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44984.log 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49134.log.gz 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48510.log 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44469.log: /var/log/ceph/ceph-client.admin.44984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45951.log 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44469.log.gz 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48510.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44984.log.gz 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45267.log 2024-03-07T07:21:10.125 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/ceph-client.admin.45951.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48510.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47277.log 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45951.log.gz 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45267.log.gz 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45393.log 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-03-07T07:21:10.126 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45393.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.47277.log.gz -- 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/ceph-client.admin.47067.log 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45393.log.gz 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52067.log 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44987.log 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52067.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47067.log.gz 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47151.log 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44987.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52067.log.gz 2024-03-07T07:21:10.127 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-03-07T07:21:10.128 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44987.log.gz 2024-03-07T07:21:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43965.log 2024-03-07T07:21:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52235.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47151.log.gz 2024-03-07T07:21:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2024-03-07T07:21:10.128 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-03-07T07:21:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51647.log 2024-03-07T07:21:10.128 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44007.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43965.log.gz 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48336.log 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51647.log.gz/var/log/ceph/ceph-mgr.y.log: 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52109.log 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48336.log.gz 2024-03-07T07:21:10.129 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.52109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46295.log 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52109.log.gz 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr: -5/var/log/ceph/ceph-client.admin.48252.log: --verbose -- /var/log/ceph/ceph-client.admin.44982.log 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.0.46295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46295.log.gz 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50364.log 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44427.log 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44982.log.gz/var/log/ceph/ceph-client.admin.50364.log: 2024-03-07T07:21:10.130 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50364.log.gz 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44427.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48924.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44427.log.gz 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48798.log 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.0.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48882.log 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-07T07:21:10.131 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45519.log 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48798.log.gz 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48882.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45519.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48882.log.gz 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44385.log 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45519.log.gz 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45645.log 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2024-03-07T07:21:10.132 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45645.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44385.log.gz 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45645.log.gz 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44649.log: /var/log/ceph/ceph-client.admin.43881.log: gzip -5 --verbose 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr: -- /var/log/ceph/ceph-client.admin.44651.log 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-07T07:21:10.133 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45771.log 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44651.log.gz 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47529.log 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.45771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45771.log.gz/var/log/ceph/ceph-client.admin.47529.log: 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48468.log 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47529.log.gz/var/log/ceph/ceph-client.admin.49050.log: 94.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:21:10.134 INFO:teuthology.orchestra.run.smithi064.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46983.log 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49969.log 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.46983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48840.log 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48468.log.gz 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6%/var/log/ceph/ceph-client.admin.49969.log: -- replaced with /var/log/ceph/ceph-client.admin.46983.log.gz 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48840.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49969.log.gz 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48840.log.gz 2024-03-07T07:21:10.135 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45687.log 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49716.log 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47571.log: /var/log/ceph/ceph-client.admin.45687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47319.log 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.49716.log: 87.7% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45687.log.gz -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 88.4% --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49716.log.gz 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47445.log 2024-03-07T07:21:10.136 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6%/var/log/ceph/ceph-client.admin.44133.log: -- replaced with /var/log/ceph/ceph-client.admin.47319.log.gz 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47790.log 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43733.log 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.47445.log: /var/log/ceph/ceph-client.admin.47790.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47790.log.gz 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47445.log.gz 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50053.log 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50011.log 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50053.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43733.log.gz 2024-03-07T07:21:10.137 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52361.log 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50053.log.gz 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51270.log 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.50011.log: /var/log/ceph/ceph-client.admin.52361.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50011.log.gz 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52361.log.gz 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43222.log 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.51270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48048.log 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43222.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51270.log.gz 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45309.log 2024-03-07T07:21:10.138 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43222.log.gz 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44986.log 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.48048.log: /var/log/ceph/ceph-client.admin.45309.log: 88.3% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45309.log.gz -- replaced with /var/log/ceph/ceph-client.admin.48048.log.gz 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.44986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43821.log 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-mds.a.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44986.log.gz 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/ceph-client.admin.43821.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-07T07:21:10.139 INFO:teuthology.orchestra.run.smithi064.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43821.log.gz 2024-03-07T07:21:10.140 INFO:teuthology.orchestra.run.smithi120.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-07T07:21:10.142 INFO:teuthology.orchestra.run.smithi120.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-07T07:21:10.151 INFO:teuthology.orchestra.run.smithi064.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-07T07:21:10.159 INFO:teuthology.orchestra.run.smithi120.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-07T07:21:10.162 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-07T07:21:10.171 INFO:teuthology.orchestra.run.smithi120.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-07T07:21:10.178 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-07T07:21:10.181 INFO:teuthology.orchestra.run.smithi120.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-07T07:21:10.182 INFO:teuthology.orchestra.run.smithi120.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-07T07:21:10.184 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-03-07T07:21:10.184 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m0.156s 2024-03-07T07:21:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:user 0m0.786s 2024-03-07T07:21:10.185 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m0.050s 2024-03-07T07:21:10.186 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-07T07:21:10.188 INFO:teuthology.orchestra.run.smithi064.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-07T07:21:10.189 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-03-07T07:21:10.190 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m0.200s 2024-03-07T07:21:10.190 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m0.781s 2024-03-07T07:21:10.190 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m0.139s 2024-03-07T07:21:10.190 INFO:tasks.ceph:Archiving logs... 2024-03-07T07:21:10.190 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/remote/smithi064/log 2024-03-07T07:21:10.191 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-07T07:21:10.480 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/remote/smithi120/log 2024-03-07T07:21:10.481 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-07T07:21:10.611 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-07T07:21:10.624 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-07T07:21:10.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-07T07:21:10.627 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-07T07:21:10.708 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-07T07:21:10.723 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-07T07:21:10.723 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:21:10.726 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T07:21:10.738 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:21:10.739 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-03-07T07:21:10.742 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:21:10.742 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-03-07T07:21:10.742 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 24 +2363us[+2359us] +/- 112ms 2024-03-07T07:21:10.743 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 6 377 23 -934us[ -938us] +/- 64ms 2024-03-07T07:21:10.743 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -473us[ -477us] +/- 75ms 2024-03-07T07:21:10.743 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:21:10.744 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T07:21:10.744 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-03-07T07:21:10.744 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +2352us[+2356us] +/- 112ms 2024-03-07T07:21:10.744 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -943us[ -940us] +/- 64ms 2024-03-07T07:21:10.744 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -503us[ -503us] +/- 75ms 2024-03-07T07:21:10.745 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T07:21:10.745 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-07T07:21:10.754 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-07T07:21:10.755 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-07T07:21:10.764 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-07T07:21:10.853 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-07T07:21:10.919 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:21:10.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:21:10.948 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T07:21:10.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:21:10.978 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-07T07:21:10.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-07T07:21:10.995 INFO:teuthology.task.internal:Duration was 554.094987 seconds 2024-03-07T07:21:10.996 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-07T07:21:11.005 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-07T07:21:11.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-07T07:21:11.008 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-07T07:21:11.048 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:21:11.060 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T07:21:11.395 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-07T07:21:11.395 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-03-07T07:21:11.396 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-07T07:21:11.455 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-03-07T07:21:11.455 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-07T07:21:11.486 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-07T07:21:11.487 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-07T07:21:11.497 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-07T07:21:11.555 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-07T07:21:11.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-07T07:21:11.565 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-07T07:21:11.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-07T07:21:11.702 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-07T07:21:11.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-07T07:21:11.728 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-07T07:21:11.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-07T07:21:11.766 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:21:11.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:21:11.793 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-03-07T07:21:11.821 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-03-07T07:21:11.836 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:21:11.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:21:11.858 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-07T07:21:11.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T07:21:11.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-07T07:21:11.902 INFO:teuthology.task.internal:Transferring archived files... 2024-03-07T07:21:11.902 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/remote/smithi064 2024-03-07T07:21:11.903 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-07T07:21:11.939 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961/remote/smithi120 2024-03-07T07:21:11.939 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-07T07:21:11.974 INFO:teuthology.task.internal:Removing archive directory... 2024-03-07T07:21:11.974 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-07T07:21:11.976 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-07T07:21:12.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-07T07:21:12.035 INFO:teuthology.task.internal:Not uploading archives. 2024-03-07T07:21:12.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-07T07:21:12.045 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-07T07:21:12.045 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-07T07:21:12.048 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-07T07:21:12.059 INFO:teuthology.orchestra.run.smithi064.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 7 07:21 /home/ubuntu/cephtest 2024-03-07T07:21:12.077 INFO:teuthology.orchestra.run.smithi120.stdout: 265313 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 7 07:21 /home/ubuntu/cephtest 2024-03-07T07:21:12.079 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-07T07:21:12.090 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-07T07:21:12.110 INFO:teuthology.nuke:Checking targets against current locks 2024-03-07T07:21:12.136 DEBUG:teuthology.nuke:shortname: smithi064 2024-03-07T07:21:12.136 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T07:21:12.164 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:02:47.672493', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2bcusr4TQ8ifT++ozEtIvUg06HaYDkl/6AuNcvne7Y4djc9CLlGseR6b6eKg71QjAdz5ePrAZ1Uh6d2DlYMC4='} 2024-03-07T07:21:12.165 DEBUG:teuthology.nuke:shortname: smithi120 2024-03-07T07:21:12.165 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T07:21:12.187 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7583961', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 07:02:47.670462', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJJs1ChUY4bEZKgqqgA0akQBieTGRewUeO2K2XEDMuldIsU4t355t0ku+Wta4XCmxCzmq+2kp7/L7lbr5KyBHDo='} 2024-03-07T07:21:12.220 INFO:teuthology.orchestra.console.smithi064:Power off 2024-03-07T07:21:12.221 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T07:21:12.240 INFO:teuthology.orchestra.console.smithi120:Power off 2024-03-07T07:21:12.240 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T07:21:12.253 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2024-03-07T07:21:12.254 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:21:12.271 DEBUG:teuthology.orchestra.console.smithi120:power off output: Chassis Power Control: Down/Off 2024-03-07T07:21:12.271 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:21:12.290 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-03-07T07:21:12.304 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is on 2024-03-07T07:21:16.292 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:21:16.306 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:21:16.404 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-03-07T07:21:16.415 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is on 2024-03-07T07:21:20.406 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:21:20.416 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T07:21:20.517 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2024-03-07T07:21:20.525 DEBUG:teuthology.orchestra.console.smithi120:check power output: Chassis Power is off 2024-03-07T07:21:20.619 INFO:teuthology.orchestra.console.smithi064:Power off completed 2024-03-07T07:21:20.626 INFO:teuthology.orchestra.console.smithi120:Power off completed 2024-03-07T07:21:20.731 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/xfstests-dev} duration: 554.0949866771698 failure_reason: 'Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=cad744c4d5c7483f862069efe14b291f status: fail success: false 2024-03-07T07:21:20.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T07:21:20.816 INFO:teuthology.run:FAIL